12. ARCore Connect API

ARCore can be connected with ARGear and be used. Once connected with ARGear, you can use both of ARCore and ARGear functions such as item, beauty, and bulge.
For the details of ARCore, please refer to “https://developers.google.com/ar/develop".
​
12.1 Configuration Settings
Additional settings are required after "1. Configuration Settings" step as below.
<build.gradle settings>
defaultConfig {
​
    minSdkVersion 24
}
​
dependencies {
   implementation "com.google.ar:core:1.13.0"
   implementation "de.javagl:obj:0.2.1"
}
<AndroidManifest.xml settings>
<application>
   ...
    <meta-data
        android:name="com.google.ar.core"
        android:value="required" />
   …
</application>
​
12.2 Session Creation and Execution
Both of ARGSession and ARCore Session need to be created and executed together. For the ARGSession, please refer to "3. ARGSession" chapter.
When creating ARGSession, ARGInferenceConfig.Feature needs to be set to "EXT_ARCORE_FACE_TRACKING".
A sample code for Session creation and execution is written below.
<ARGSession Creation>
private void createSession() {
​
    // create ARGear session
    ARGConfig argConfig
            = new ARGConfig(AppConfig.API_URL, 
                                          AppConfig.API_KEY, 
                                          AppConfig.SECRET_KEY, 
                                          AppConfig.AUTH_KEY);
    Set<ARGInferenceConfig.Feature> inferenceConfig
            = EnumSet.of(ARGInferenceConfig.Feature.EXT_ARCORE_FACE_TRACKING);
​
    argSession = new ARGSession(this, argConfig, inferenceConfig);
}
<ARCore Session Creation/Execution & ARGSession Execution>
protected void onResume() {
    super.onResume();
​
    // argesr session execution
    argsession.resume();
​
    // arcore session creation
    arcoreSession = new Session(/* context= */ activity, EnumSet.of(Session.Feature.FRONT_CAMERA));
    Config config = new Config(session);
    config.setAugmentedFaceMode(Config.AugmentedFaceMode.MESH3D);
    arcoreSession.configure(config);
​
    // arcore session execution
    arcoreSession.resume();
}
​
12.3 Rendering
In order to draw a result frame, ARCore session update must be called in advance and texture ID used for the update should be fed to ARGear. Moreover, mesh vertices and matrix obtained from ARCore session should be fed to ARGear.
Once ARGSession.drawFrame is called, rendering starts with fed information from ARCore. If Item, Filter, or Beauty is set (Reference : 8. Set Contents), the effect will be rendered together and ARGFrame will be returned as its result.
The ARGFrame contains the final rendered texture ID and you can draw the frame on screen by using this texture ID.
Below is a sample rendering code.
<Rendering Example Code>
ArrayList<FloatBuffer> verticesList = new ArrayList<>();
ArrayList<float []> poseMatrixList = new ArrayList<>();
float[] mProjectionMatrix = new float[16];
float[] mViewMatrix = new float[16];
​
public void onDrawFrame(GL10 gl) {
​
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
​
    arcoreSession.setCameraTextureName(backgroundRenderer.getTextureId());
​
    // update arcore session
    Frame frame = arcoreSession.update();
​
    if (frame == null || frame.getCamera() == null) return;
​
    if (frame.getTimestamp() == 0) return;
​
    if (arcoreSession.isAugmentedFaceMode()) {
        verticesList.clear();
        poseMatrixList.clear();
​
        for (AugmentedFace face : arcoreSession.getSession().getAllTrackables(AugmentedFace.class)) {
            if (face.getTrackingState() == TrackingState.TRACKING) {
                // mesh vertices
                FloatBuffer faceVertices = face.getMeshVertices();
                verticesList.add(faceVertices);
​
               // center and region poses
                float[] facePoseMatrix = new float[16];
                Pose facePose = face.getCenterPose();
                facePose.toMatrix(facePoseMatrix, 0);
                poseMatrixList.add(facePoseMatrix);
            }
        }
​
        // arcore camera projection matrix
        frame.getCamera().getProjectionMatrix(mProjectionMatrix, 0, 0.1f, 100.0f);
​
        // arcore camera view matrix
        frame.getCamera().getViewMatrix(mViewMatrix, 0);
​
        Size textureSize = arcoreSession.getSession().getCameraConfig().getTextureSize();
        if (mTextureSize == null || !mTextureSize.equals(textureSize)) {
            mTextureSize = textureSize;
            mARGSession.setCameraConfig(new ARGCameraConfig(textureSize.getWidth(),
                    textureSize.getHeight(),
                    0,
                    0,
                    0,
                    true,
                    0));
        }
​
        // feed mesh vertices, pose matrix, projection matrix, view matrix
        mARGSession.applyAdditionalFaceInfo(verticesList, poseMatrixList, mProjectionMatrix, mViewMatrix);
​
        // feed texture id
        mARGSession.feedTexture(arcoreSession.getTextureId(), mTextureSize);
​
        // render argear frame
        ARGFrame argFrame = mARGSession.drawFrame(gl, mScreenWidth, mScreenHeight);
​
        // render screen
        mScreenRenderer.draw(argFrame, mScreenWidth, mScreenHeight, mTextureSize.getWidth(), mTextureSize.getHeight());
    }
}
​
​
12.4 Set Contexts
Please refer to the chapter "8. Set Contents".
​
​
​
​
​
Android - Previous
11. Enable Debugging Mode
Next - Android
13. API Reference
Last updated 2 weeks ago