12. ARKit Connect API

ARKit can be connected to ARGear. Once connected, you can utilize functions in both of ARKit and ARGear (item, beauty, and bulge) together.
We provide Swift based sample codes. For the details of ARKit, please refer to “https://developer.apple.com/documentation/arkit”.
12.1 Configuration
Once 1.Configuration Settings is done, you should import ARKit.
import ARKit
12.2 Session Creation and Execution
Both of ARGSession and ARCore Session need to be created and executed together. For the ARGSession, please refer to 3. ARGSession chapter.
When creating ARGSession, ARGInferenceFeature needs to be set to ".extARKitFaceTracking".
A sample code below shows ARGSession creation and execution.
let config = ARGConfig(
    apiURL: API_HOST,
    apiKey: API_KEY,
    secretKey: API_SECRET_KEY,
    authKey: API_AUTH_KEY
)
​
argSession = try ARGSession(argConfig: config, feature: .extARKitFaceTracking)
argSession?.delegate = self
} catch let error as NSError {
} catch let exception as NSException {
}
​
argSession?.run()
A sample code below shows ARKit Session creation and execution.
arKitSession = ARSession()
​
let arkitFaceTrackingConfig = ARFaceTrackingConfiguration()
arKitSession?.delegate = self
arKitSession?.run(arkitFaceTrackingConfig)
12.3 Rendering
In order to draw a result frame on screen, CVPixelBufferRef and ARAnchor information in updated ARFrame from session of ARSessionDelegate need to be passed to ARGSession.
For this, applyAdditionalFaceInfo function in ARGSession should be called with transform and geometry.vertices of ARFaceAnchor, the size of result view, and ARGSessionProjectPointHandler (a converting function) as parameters.
Once didUpdate(_ arFrame: ARGFrame) function of ARGSessionDelegate is called, rendering proceeds according to provided information from ARKit.
When Item, Filter, or Beauty is set, a result ARGFrame will be rendered altogether and returned.
In ARGFrame, a final rendered result will be set in CVPixelBuffer and you can draw the result with it.
CVPixelBuffer may require rotation or flip to show a frame correctly. For the details, please refer to drawARCameraPreview() from our official sample code.
A sample code for rendering is written below.
// ARSessionDelegate
public func session(_ session: ARSession, didUpdate frame: ARFrame) {
​
    let viewportSize = view.bounds.size
    var updateFaceAnchor: ARFaceAnchor? = nil
    var isFace = false
    if let faceAnchor = frame.anchors.first as? ARFaceAnchor {
        if faceAnchor.isTracked {
            updateFaceAnchor = self.currentARKitFaceAnchor
            isFace = true
        }
    } else {
        if let _ = frame.anchors.first as? ARPlaneAnchor {
        }
    }
​
    let handler: ARGSessionProjectPointHandler = { (transform: simd_float3, orientation: UIInterfaceOrientation, viewport: CGSize) in
        return frame.camera.projectPoint(transform, orientation: orientation, viewportSize: viewport)
    }
        
    if isFace {
        if let faceAnchor = updateFaceAnchor {
            self.argSession?.applyAdditionalFaceInfo(withPixelbuffer: frame.capturedImage, transform: faceAnchor.transform, vertices: faceAnchor.geometry.vertices, viewportSize: viewportSize, convert: handler)
        } else {
            self.argSession?.feedPixelbuffer(frame.capturedImage)
        }
    } else {
            self.argSession?.feedPixelbuffer(frame.capturedImage)
    }
}
​
// ARGSessionDelegate
public func didUpdate(_ arFrame: ARGFrame) {
    guard let _ = arKitSession.configuration as? ARFaceTrackingConfiguration else  {
        return
    }
​
    if let cvPixelBuffer = arFrame.renderedPixelBuffer {
        // draw cvPixelBuffer
    }
}
12.4 Set Contents
Please refer to the chapter, 8. Set Contents.
iOS - Previous
11. Enable Debugging Mode
Next - iOS
13. API Reference
Last updated 2 weeks ago