13. ARCore Connect API

Search…
ARCore can be connected and used with ARGear. Once connected with ARGear, you can use both ARCore and ARGear functions such as items, beauty, and bulge.
For the details of ARCore, please refer to “https://developers.google.com/ar/develop".
​
13.1 Configuration Settings
Additional settings are required after "1. Configuration Settings" step as below.
In order to connect with ARCore, argear-renderer.aar instead of argear.aar needs to be used.
<build.gradle settings>
1
defaultConfig {
2
​
3
    minSdkVersion 24
4
}
5
​
6
dependencies {
7
   implementation "com.google.ar:core:1.17.0"
8
   implementation "de.javagl:obj:0.2.1"
9
   implementation (name: 'argear-renderer', ext: 'aar')
10
​
11
}
Copied!
<AndroidManifest.xml settings>
1
<application>
2
   ...
3
    <meta-data
4
        android:name="com.google.ar.core"
5
        android:value="required" />
6
   …
7
</application>
Copied!
​
13.2 Session Creation and Execution
Both the ARGSession and ARCore Session need to be created and executed together. For the ARGSession, please refer to chapter 3. ARGSession.
When creating ARGSession, the ARGInferenceConfig.Feature setting is not required for ARCore.
Sample code for Session creation and execution is written below.
<ARGSession Creation>
1
private void createSession() {
2
​
3
    // create ARGear session
4
    ARGConfig argConfig
5
            = new ARGConfig(AppConfig.API_URL, 
6
                                          AppConfig.API_KEY, 
7
                                          AppConfig.SECRET_KEY, 
8
                                          AppConfig.AUTH_KEY);
9
    argSession = new ARGSession(this, argConfig);
10
}
11
​
Copied!
<ARCore Session Creation/Execution & ARGSession Execution>
1
private void createSession() {
2
​
3
    // create ARCore session
4
    arcoreSession = new ARCoreSession(this);
5
​
6
    // create ARGear session
7
    ARGConfig argConfig
8
            = new ARGConfig(AppConfig.API_URL, 
9
                                          AppConfig.API_KEY, 
10
                                          AppConfig.SECRET_KEY, 
11
                                          AppConfig.AUTH_KEY);
12
    argsession = new ARGSession(this, argConfig);
13
}
14
​
15
protected void onCreate(Bundle savedInstanceState) {
16
    ...
17
    createSession()
18
}
19
​
20
protected void onResume() {
21
    super.onResume();
22
​
23
    argsession.resume();
24
​
25
    arcoreSession.resume(this);
26
}
27
​
Copied!
​
13.3 Rendering
In order to draw a resulting frame, ARCore session update must be called in advance and the texture ID used for the update should be fed to ARGear. Moreover, mesh vertices and matrix obtained from ARCore session should be fed to ARGear.
Once ARGSession.drawFrame is called, rendering starts with fed information from ARCore. If Item, Filter, or Beauty is set (Reference: 8. Set Contents), the effect will be rendered together with the ARCore frame and ARGFrame will be returned as the result.
The ARGFrame contains the final rendered texture ID and you can draw the frame on screen using this texture ID.
Below is a sample rendering code.
<Rendering Example Code>
1
ArrayList<FloatBuffer> verticesList = new ArrayList<>();
2
ArrayList<float []> poseMatrixList = new ArrayList<>();
3
float[] mProjectionMatrix = new float[16];
4
float[] mViewMatrix = new float[16];
5
​
6
public void onDrawFrame(GL10 gl) {
7
​
8
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
9
​
10
    arcoreSession.setCameraTextureName(backgroundRenderer.getTextureId());
11
​
12
    // update arcore session
13
    Frame frame = arcoreSession.update();
14
​
15
    if (frame == null || frame.getCamera() == null) return;
16
​
17
    if (frame.getTimestamp() == 0) return;
18
​
19
    if (arcoreSession.isAugmentedFaceMode()) {
20
        verticesList.clear();
21
        poseMatrixList.clear();
22
​
23
        for (AugmentedFace face : arcoreSession.getSession().getAllTrackables(AugmentedFace.class)) {
24
            if (face.getTrackingState() == TrackingState.TRACKING) {
25
                // mesh vertices
26
                FloatBuffer faceVertices = face.getMeshVertices();
27
                verticesList.add(faceVertices);
28
​
29
               // center and region poses
30
                float[] facePoseMatrix = new float[16];
31
                Pose facePose = face.getCenterPose();
32
                facePose.toMatrix(facePoseMatrix, 0);
33
                poseMatrixList.add(facePoseMatrix);
34
            }
35
        }
36
​
37
        // arcore camera projection matrix
38
        frame.getCamera().getProjectionMatrix(mProjectionMatrix, 0, 0.1f, 100.0f);
39
​
40
        // arcore camera view matrix
41
        frame.getCamera().getViewMatrix(mViewMatrix, 0);
42
​
43
        Size textureSize = arcoreSession.getSession().getCameraConfig().getTextureSize();
44
        if (mTextureSize == null || !mTextureSize.equals(textureSize)) {
45
            mTextureSize = textureSize;
46
            mARGSession.setCameraConfig(new ARGCameraConfig(textureSize.getWidth(),
47
                    textureSize.getHeight(),
48
                    0,
49
                    0,
50
                    0,
51
                    true,
52
                    0));
53
        }
54
​
55
        // feed mesh vertices, pose matrix, projection matrix, view matrix
56
        mARGSession.applyAdditionalFaceInfo(verticesList, poseMatrixList, mProjectionMatrix, mViewMatrix);
57
​
58
        // feed texture id
59
        mARGSession.feedTexture(arcoreSession.getTextureId(), mTextureSize);
60
​
61
        // render argear frame
62
        ARGFrame argFrame = mARGSession.drawFrame(gl, mScreenWidth, mScreenHeight);
63
​
64
        // render screen
65
        mScreenRenderer.draw(argFrame, mScreenWidth, mScreenHeight, mTextureSize.getWidth(), mTextureSize.getHeight());
66
    }
67
}
68
​
Copied!
​
13.4 Set Contents
Please refer to the chapter 8. Set Contents.
​
​
​
​
​
Android - Previous
12. Enable Debugging Mode
Next - Android
14. API Reference
Last modified 1yr ago
Copy link
Contents
13.1 Configuration Settings
13.2 Session Creation and Execution
13.3 Rendering
13.4 Set Contents