TEEP P. Yang Internet-Draft China Mobile Intended status: Standards Track T. Pang Expires: 8 January 2024 Huawei Technology Co.,Ltd. X. Zhang AntGroup 7 July 2023 TEE Distributed Provisioning Relay draft-yang-teep-tee-dpr-00 Abstract Big data computing framework uses Master-Worker structure to provision applications and process data. When process remote attestation for this kind of framework in TEE cluster, Verifier or Relying party needs to know the dynamic reference value of Worker nodes. This document shows how to use Master to relay Worker provision information to Verifier or Relying Party and its relevant protocol. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 8 January 2024. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights Yang, et al. Expires 8 January 2024 [Page 1] Internet-Draft TEE-DPR July 2023 and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Architecture . . . . . . . . . . . . . . . . . . . . . . . . 3 5. Worker Runtime Provisioning Relay . . . . . . . . . . . . . . 4 5.1. Rationality . . . . . . . . . . . . . . . . . . . . . . . 5 5.2. Worker Runtime Provisioning Relay by CDDL . . . . . . . . 5 5.2.1. Task serialized . . . . . . . . . . . . . . . . . . . 6 5.2.2. Worker Hardware Manifest . . . . . . . . . . . . . . 6 5.2.3. Worker Software Manifest . . . . . . . . . . . . . . 7 6. Security Considerations . . . . . . . . . . . . . . . . . . . 7 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 8.1. Normative References . . . . . . . . . . . . . . . . . . 7 8.2. Informative References . . . . . . . . . . . . . . . . . 8 Appendix A. Appendix 1 Full CDDL of TEE DP protocol . . . . . . 9 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 1. Introduction In big data area, different from stand-alone application, distributed application like machine learning always need to be split by Master- Worker framework like MESOS[MESOS], YARM[YARN], Spark[Spark] or kubernets[Kubernetes] . The Master node in charges of splitting distributed application to computing tasks. Then these tasks will be deployed on Workers for execution. TEE could be used to protect the application and its secret data, if only the whole process lifecycle is covered by TEE cluster. If a Verifier or Relying Party would like to attest distributed application running in TEE cluster, it has to verify the Worker nodes since the Worker nodes are the real place in where specific tasks and data are processing. The Master node conducts the splitting process and provision specific tasks or executable code to Workers after running the main part of application. Master node could relay these tasks or executable code information to Verifier or Relying party for remote attestation. As a result, Verifier or Relying Party could assess the trustworthiness of the lifecycle of distributed application in TEE cluster. This document specifies the method and protocol of this distributed TEE provision relay. Yang, et al. Expires 8 January 2024 [Page 2] Internet-Draft TEE-DPR July 2023 2. Terminology The following terms are used: * Master: The node that runs the main application and in charges of splitting and distributing tasks to Worker nodes. * Worker: The node that runs the specific tasks that generated by Master. * Relay: Master transfer the original provision message to Verifier/ Relying party for remote attestation. 3. Use Cases In privacy-preserve computing, participants want to create a unified machine learning model without leaking private data owned by each other. TEE cluster as a trusted party could make sure data inside this environment is integrated and confidential. If the federated machine learning participants trust this TEE cluster and application, they could transfer their data in that TEE cluster and generate the final machine learning model. The method and protocol described in this document could help privacy-preserve computing participants to process remote attestation of TEE cluster during runtime. 4. Architecture The following figure shows the architecture of TEE distributed provisioning relay. In this architecture, Master runs the main part of distributed application, which will generate tasks or executable code and provision to Workers. Master accepts remote attestation challenge from Verifier and response Evidence of Master. Then, Master relay provisioning information of Workers to Verifier/Relying Party. In the end, Evidence of Workers will be transfered to Verifier/Relying Party by Master. The security or trust logic of this architecture is to build a trust chain between Master and Workers, in which the Verifier/Relying party could trust the Master first and let the Master to relay the provisioning information of Workers to Verifier/Relying Party. Yang, et al. Expires 8 January 2024 [Page 3] Internet-Draft TEE-DPR July 2023 +-------------+ | Verifier/ | |Relying Party| +-------------+ ^ 1. Remote attestation | 2. Return Evidence of Master of Master | 4. Relay Worker runtime provisioning v 6. Return Evidence of Workers +-----------------------+ |Master/Main application| +-----------------------+ 3. Generate tasks & ^ 5. Remote attestation of Workers distribute to Workers | ---------------------+---------------------- | | | v v v +--------+ +--------+ +--------+ | Worker | | Worker | | Worker | +--------+ +--------+ +--------+ Figure 1: Distributed Application Remote Attestation The steps of TEE distributed provisioning relay is described below. - Verifier/Relying party sends challenge to Master and tries to assess the trustworthiness of this distributed application runs in TEE cluster. - Master generates Evidence of itself and return to Verifier/Relying Party. - Master generates tasks and provisions to Workers. - Meanwhile, Master transfer the tasks provisioning information and Worker information back to Verifier/Relying Party to generate reference value of Workers. - Master challenges Worker for Evidence. - Master sends the Evidence of Workers back to Verifier/Relying Party for attestation. 5. Worker Runtime Provisioning Relay Worker runtime provisioning relay message is sent from Master to Verifier/Relying Party. It's function is to provide Verifier/Relying Party information to generate reference value of Workers. This message can only be sent after remote attestation of Master. Yang, et al. Expires 8 January 2024 [Page 4] Internet-Draft TEE-DPR July 2023 5.1. Rationality In big data computing framework, Reference Value of Workers may change during runtime based on computing stages and input data source. Thus it is not rational to generate Reference Value of Workers statically. Instead, Master is the manage and monitor center of the tasks in Workers. The following reason shows that Master is the reasonable choice to generate and relay Worker runtime provisioning message. * Master runs the main part of distributed application which reference value is static and could be pre-generated. * Master is task generator, it in charges of generating different tasks or executable code for Workers in different processing stage. * Master is computing resource manager of TEE cluster, it in charges of orchestrating TEE hardware resource. 5.2. Worker Runtime Provisioning Relay by CDDL Worker runtime provisioning relay message includes 3 message type, task-serialized, hardware-manifest and software-manifest. The following figure shows the message framework of Worker runtime provisioning relay in CDDL. tee-dpr-message = $tee-dpr-message-type .within tee-dpr-message-framework tee-dpr-message-framework = [ type: $tee-dpr-type, optionis: { * tee-dpr-option} ] tee-dpr-option = (uint =>any) $tee-dpr-message-type /= tee-task-serialized $tee-dpr-message-type /= tee-worker-hardware-manifest $tee-dpr-message-type /= tee-worker-software-manifest $tee-dpr-type = uint .size 1 TEE-task-serialized = 1 TEE-worker-hardware-manifest = 2 TEE-worker-software-manifest = 3 Figure 2: TEE Distributed Provisioning Relay Message Framework Yang, et al. Expires 8 January 2024 [Page 5] Internet-Draft TEE-DPR July 2023 5.2.1. Task serialized Task serialized represents the execution content like executable code of task. Master generate and distribute tasks to workers by its default mechanism. Meanwhile Master needs to relay this tee-task- serialized message to Verifier/Relying Party for remote attestation of Workers. tee-task-serialized = [ type: TEE-task-serialized options:{ Recommended-serialized-type: uint .bits serialized-type serialized-payload = bstr } ] serialized-type = &( json: 0 protobuf: 1 thrift: 2 pre-defined: 3 ) 5.2.2. Worker Hardware Manifest Worker hardware manifest message needs to be relayed to Verifier before remote attestation. Worker hardware manifest is intent to describe the specific hardware environment of Workers, which includes CPU, memory, and other necessary information. tee-worker-hardware-manifest = [ type: TEE-worker-hardware-manifest options:{ CPU-core-number: uint .size 1 CPU-architecture: text .size(1..128) CPU-version: text .size(1..128) CPU-frequency: unit .size 2 ;the cpu frequency unit is MHZ MEMORY-size: uint .size 4 ;the memory size unit is MB } ] Yang, et al. Expires 8 January 2024 [Page 6] Internet-Draft TEE-DPR July 2023 5.2.3. Worker Software Manifest Worker software manifest message includes operation system version, memory allocation method, memory protection method, etc. This message needs to be relayed to Verifier before remote attestation. Worker software manifest message is intent to describe the specific software environment of Worker, which includes operation system, memory, and other necessary information. Verifier will use this message to generate reference value. tee-worker-software-manifest = [ type: TEE-worker-software-manifest options:{ os-version: text .size(1..128) memory-allocation: text .size(1..128) sadr: bool canary: bool worker-executor-version: text .size(1..128) time-stamp: text .size(1..128) } ] 6. Security Considerations This protocol is used to relay Worker TEE provisioning message between Master and Verifier, between which secure channel could be built by TLS and etc. The secure channel between Worker and Verifier could also use TLS or other secure protocol to protect provisioning message. 7. IANA Considerations This document does not require actions by IANA. 8. References 8.1. Normative References [I-D.ietf-teep-protocol] Tschofenig, H., Pei, M., Wheeler, D. M., Thaler, D., and A. Tsukamoto, "Trusted Execution Environment Provisioning (TEEP) Protocol", Work in Progress, Internet-Draft, draft- ietf-teep-protocol-15, 3 July 2023, . Yang, et al. Expires 8 January 2024 [Page 7] Internet-Draft TEE-DPR July 2023 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . 8.2. Informative References [I-D.ietf-teep-architecture] Pei, M., Tschofenig, H., Thaler, D., and D. M. Wheeler, "Trusted Execution Environment Provisioning (TEEP) Architecture", Work in Progress, Internet-Draft, draft- ietf-teep-architecture-19, 24 October 2022, . [I-D.ietf-teep-usecase-for-cc-in-network] Yang, P., chenmeiling, Su, L., and T. Pang, "TEEP Usecase for Confidential Computing in Network", Work in Progress, Internet-Draft, draft-ietf-teep-usecase-for-cc-in-network- 04, 5 July 2023, . [Kubernetes] Community, K., "Kubernetes Cloud Controller Manager", 17 December 2022, . [MapReduce] Community, A. M., "MapReduce Overview", 27 July 2022, . [MESOS] Community, A. M., "MESOS Architecture", 2 March 2023, . [Spark] Community, S., "Spark Overview", 2 March 2023, . [YARN] Community, A. H., "Apache Hadoop YARN", 29 July 2022, . Yang, et al. Expires 8 January 2024 [Page 8] Internet-Draft TEE-DPR July 2023 Appendix A. Appendix 1 Full CDDL of TEE DP protocol The full CDDL of TEE distributed provisioning protocol is shown below. tee-dpr-message = $tee-dpr-message-type .within tee-dpr-message-framework tee-dpr-message-framework = [ type: $tee-dpr-type, optionis: { * tee-dpr-option} ] tee-dpr-option = (uint =>any) $tee-dpr-message-type /= tee-task-serialized $tee-dpr-message-type /= tee-worker-hardware-manifest $tee-dpr-message-type /= tee-worker-software-manifest $tee-dpr-type = uint .size 1 TEE-task-serialized = 1 TEE-worker-hardware-manifest = 2 TEE-worker-software-manifest = 3 tee-task-serialized = [ type: TEE-task-serialized options:{ Recommended-serialized-type: uint .bits serialized-type Serialized-payload = bstr } ] serialized-type = &( json: 0 protobuf: 1 thrift: 2 pre-defined: 3 ) tee-worker-hardware-manifest = [ type: TEE-worker-hardware-manifest options:{ CPU-core-number: uint .size 1 CPU-architecture: text .size(1..128) CPU-version: text .size(1..128) CPU-frequency: unit .size 2 ;the cpu frequency unit is MHZ MEMORY-size: uint .size 4 ;the memory size unit is MB } ] Yang, et al. Expires 8 January 2024 [Page 9] Internet-Draft TEE-DPR July 2023 tee-worker-software-manifest = [ type: TEE-worker-software-manifest options:{ OS-version: text .size(1..128) Memory-allocation: text .size(1..128) Sadr: bool Canary: bool Worker-executor-version: text .size(1..128) Time-stamp: text .size(1..128) } ] ; labels of mapkey for tee dp message parameters, uint (0..15) Recommended-serialized-type = 0 Serialized-payload = 1 CPU-core-number = 2 CPU-architecture = 3 CPU-version = 4 CPU-frequency = 5 MEMORY-size = 6 OS-version = 7 Memory-allocation = 8 Sadr = 9 Canary = 10 Worker-executor-version = 11 Time-stamp = 12 Authors' Addresses Penglin Yang China Mobile No.32 Xuanwumen West Street Beijing China Email: yangpenglin@chinamobile.com Ting Pang Huawei Technology Co.,Ltd. 127 Jinye Road, Yanta District Xi'an China Email: pangting@huawei.com Yang, et al. Expires 8 January 2024 [Page 10] Internet-Draft TEE-DPR July 2023 Xiaomeng Zhang AntGroup World Financial Center, No.1 East 3rd Ring Middle Road Beijing China Email: zhangxiaomeng.zxm@antgroup.com Yang, et al. Expires 8 January 2024 [Page 11]