Securing data confidentiality for on-device inference using multi-enclave trusted execution environments
摘要
On-device machine learning (ODML) is being rapidly deployed in edge-computing applications, but the on-device inference stack may still be partially untrusted (e.g., third-party model runtimes, compromised OS/frameworks), enabling exfiltration of raw inputs and inference outputs. This paper presents Split Inference, a multi-enclave trusted-execution architecture that confines the most sensitive parts of the inference pipeline to two isolated enclaves (Input Enclave and Output Enclave) while executing the compute-intensive middle layers outside enclaves for efficiency. Split Inference establishes a provable secure channels between the application and each enclave via a proposed key-distribution protocol. Our inference privacy analysis also demonstrated that the solution is immune from the label inference attack and can effectively mitigate the data reconstruction attack. We implemented Split Inference for MobileNetV1 on Keystone (a RISC-V multi-enclave TEE) and evaluate end-to-end overhead relative to a non-enclave baseline. Our experiments showed that Split Inference incurs 71% to 430% of inference overhead.