14. API Reference

This section provides a detailed description of ARGear classes.

14.1 ARGSession

ARGSession class stores statuses of an application and provides corresponding functions based on the statuses.

14.1.1 Overview

Properties

Description

NS_OPTIONS ARGInferenceFeature

ARGear Inference Option

NS_OPTIONS ARGInferenceDebugOption

ARGear Debugging Option

ARGSessionDelegate

Description

- didUpdateFrame

Called when ARGFrame is updated

Method

Description

- initWithARGConfig:error

Initialize ARGSession with ARGConfig.

- initWithARGConfig:feature:error:

Initialize ARGSession with ARGConfig and ARGInferenceFeature.

- inferenceConfig:

Configure SDK with ARGInferenceOption.

- runWithInferenceConfig:

Run ARGSession with ARGInferenceOption.

- run

Run ARGSession.

- pause

Pause ARGSession.

- updateSampleBuffer:frromConnection: *)connection

Pass sampleBuffer and connection to ARGSession.

- updateMetadataObjects:fromConnection

Pass metadataObjects and connection to ARGSession.

- applyAdditionalFaceInfoWithPixelbuffer:

transform:vertices:viewportSize:convert:

Pass additional information for face detection and CVPixelBuffer.

- feedPixelbuffer:

Pass CVPixelBuffer.

- destroy

Destroy ARGSession.

14.1.2 Details

typedef NS_OPTIONS (NSInteger,ARGInferenceFeature)

ARGInferenceFeature is a required parameter and can be obtained from ARGFrame.

Types of feature inferences are declared as follows.

typedef NS_OPTIONS (NSInteger,ARGInferenceFeature) {

ARGInferenceFeatureFaceLowTracking = 1 << 0, // Machine Learning based Tracking

ARGInferenceFeatureFaceHighTracking = 1 << 1,

ARGInferenceFeatureFaceMeshTracking = 1 << 2,

// ARGInferenceFeatureFaceBlendShapes = 1 << 3,

// ARGInferenceFeatureSegmentationHalf = 1 << 4,

// ARGInferenceFeatureSegmentationFull = 1 << 5,

// ARGInferenceFeatureHandTracking = 1 << 6,

// ARGInferenceFeatureHandLandmark2D = 1 << 7,

// ARGInferenceFeatureHandLandmark3D = 1 << 8,

// ARGInferenceFeatureBodyTracking = 1 << 9,

// ARGInferenceFeatureBodyPose2D = 1 << 10,

// ARGInferenceFeatureBodyPose3D = 1 << 11,

ARGInferenceFeatureExtARKitFaceTracking = 1 << 12 };

typedef NS_OPTIONS (NSInteger,ARGInferenceDebugOption)

Types of Debugging Options in ARGear.

ARGInferenceOptionDebugNON: Turn off Debugging Mode

ARGInferenceOptionDebugFaceTracking: Draw Face Tracking Boxes for Debugging

ARGInferenceOptionDebugFaceLandmark2D: Draw Face Landmark for Debugging

- (void)didUpdateFrame:(ARGFrame *)frame

Parameters

  • ARGFrame frame : Returned Next Frame from ARGear

- (instancetype _Nullable)initWithARGConfig:(ARGConfig*)argconfig error:(NSError **)error

Parameters

  • ARGConfig argconfig : ARGear Configuration Used during ARGSession Initialization

  • NSError error: Value when an Error Occurred

Returns

  • Created ARGSession

- (instancetype _Nullable)initWithARGConfig:(ARGConfig)argconfig feature:(ARGInferenceFeature)feature error:(NSError *)error

Initialize ARGSession with ARGConfig and ARGInferenceFeature. When an error occurs, return an error in NSError type.

Parameters

  • ARGConfig argconfig : ARGConfig for ARGSession initialization

  • ARGInferenceFeature: An option for initialization. Multiple options can be set together.

  • NSError error: When an error occurs, it contains a corresponding value.

Returns

  • 생성된 ARGSession

- (void)inferenceConfig:(ARGInferenceOption) inferenceConfig

Determine which types of deep learning features in ARGear using ARGInferenceOption.

Parameters

  • ARGInferenceOption inferenceConfig : Types of Deep Learning Features to Be Used in ARGear

- (void)runWithInferenceConfig:(ARGInferenceOption)inferenceConfig

Run ARGSession with ARGInferenceOption.

Parameters

  • ARGConfig argconfig : Configuration Options When Running ARGSession

- (void)run

Start or Resume ARGSession.

- (void)pause

Pause ARGSession.

- (BOOL)updateSampleBuffer:(CMSampleBufferRef)sampleBuffer

fromConnection:(AVCaptureConnection *)connection

Update CMSampleBufferRef and AVCaptureConnection Information Received from Camera.

Parameters

  • CMSampleBufferRef sampleBuffer : Video Frames Sample Buffer from Camera

  • AVCaptureConnection connection : Camera Connection Information

Returns

  • BOOL: Check Whether Data Is Fed to ARGear Properly

- (BOOL)updateMetadataObjects:(NSArray<__kindof AVMetadataObject *> *)metadataObjects

fromConnection:(AVCaptureConnection *)connection

Update AVMetadataObject and AVCaptureConnection Information Fed from Camera.

Parameters

  • NSArray<__kindof AVMetadataObject *> metadataObjects : An AVMetadataObject Array Containing Face Information

  • AVCaptureConnection connection : Camera Connection Information

Returns

  • BOOL: Check Whether Data is Fed to ARGear properly or not.

- (BOOL)applyAdditionalFaceInfoWithPixelbuffer:(CVPixelBufferRef)pixelbuffer

transform:(simd_float4x4)transform

vertices:(const simd_float3 *)vertices

viewportSize:(CGSize)viewportSize

convert:(ARGSessionProjectPointHandler)convertHandler

Pass CVPixelBuffer and ARAnchor information, and converting handler from ARKit.

Parameters

  • CVPixelBufferRef pixelbuffer : ARKit's ARFrame.capturedImage

  • simd_float4x4 transform: ARKit's ARFaceAnchor.transform

  • const simd_float3 *vertices: ARKit's ARFaceAnchor.geometry.vertices

  • CGSize viewportSize: size of view to be drawn from CVPixelBuffer

  • ARGSessionProjectPointHandler convertHandler: a convert handler function that passes transform, orientation, and viewport information to ARFrame.camera.projectPoint function and returns converted coordinates information.

Returns

  • BOOL: Check whether ARGear receive data properly.

- (void)feedPixelbuffer:(CVPixelBufferRef)pixelbuffer

Feed CVPixelBuffer from ARKit.

Parameters

  • CVPixelBufferRef pixelbuffer : ARFrame.capturedImage of ARKit.

- (void)destroy

Destroy ARGSession.

14.2 ARGAuth

14.2.1 Overview

Structs

Description

struct ARGAuthCallback

Returned Structure after Auth Function Called

Method

Description

- isValid

Return whether the Current Auth of ARGSession is Valid

- requestSignedUrlWithUrl:itemTitle:itemType:completion:

Request SignedURL to Download Contents.

14.2.2 Details

typedef struct ARGAuthCallback {

void (^ _Nullable Success) (NSString * _Nullable url);

void (^ _Nullable Error) (ARGStatusCode code);

} ARGAuthCallback;

Callback function called when requesting Auth information. On success, it returns URL through Success function. On failure, it returns ARGStatusCode through Error function.

Blocks

  • void (^ _Nullable Success) (NSString * _Nullable url) : When requesting Auth information succeeds, URL is returned.

  • void (^ _Nullable Error) (ARGStatusCode code) : When requesting Auth information fails, ARGStatusCode is returned.

- (BOOL)isValid

Return whether the Current Stored Auth Information is Valid.

Returns

  • BOOL: Return if the Current Auth Information is Valid.

- (void)requestSignedUrlWithUrl:(NSString *)url itemTitle:(NSString *)title itemType:(NSString *)type completion:(ARGAuthCallback)completion

To Obtain Signed URL to Download Contents, the function passes URL, title, and type information and get the result through ARGAuthCallback.

Parameters

  • NSString url : URL of Content

  • NSString title: Title of Content

  • NSString type: Type of Content

  • ARGAuthCallback completion: Callback Block Returning the Result

14.3 ARGFrame

14.3.1 Overview

Properties

Description

ARGFaces faces

Recognized Faces

CVPixelBufferRef capturedPixelBuffer

pixelBuffer of Camera Frames Fed to ARGSession

CVPixelBufferRef renderedPixelBuffer

pixelBuffer of Contents Rendered Camera Frames Fed to ARGSession

Method

Description

- displayTransform

Return the Current Camera Transform Information.

14.3.2 Details

readonly ARGFaces *faces

An Array Containing the Current Recognized Faces.

readonly CVPixelBufferRef capturedPixelBuffer

Camera pixelBuffer Fed to ARGSession. Pure Camera Frames.

readonly CVPixelBufferRef renderedPixelBuffer

Contents Rendered Camera pixelBuffer Fed to ARGSession.

- (CGAffineTransform)displayTransform

Return Transform Information of Being Rendered in Display in CGAffineTransform Instance.

Returns

  • CGAffineTransform: Transform Information Being Rendered in Display

14.4 ARGContents

ARGContents are created during ARGSession initialization. It is used to control effects of Stickers, Filters, Face Beautifications, and Face Fun Bulges.

14.4.1 Overview

Method

Description

- setItemWithType:withItemFilePath:withItemID:::

Set Contents.

- setFilterLevel:

Set Level of a Filter.

- setDefaultBeauty

Set Default Face Beautification Level Values.

- setBeautyOn:

Set a Face Beautification.

- setBeauty:value:

Set Level of a Face Beautification.

- getBeautyValue:

Get the Current Face Beautification Level.

- setBeautyValues:

Set Level of Face Beautifications at Once.

- setBulge:

Set a Face Fun Bulge Effect.

- clear:

Clear Contents.

14.4.2 Details

- (void)setItemWithType:(ARGContentItemType)itemType

withItemFilePath:(NSString * _Nullable)itemFilePath

withItemID:(NSString * _Nullable)itemId

completion:(ARGContentsCallback _Nullable)callback;

Set Type, File Path, and uuid of Contents to Be Rendered in Display.

ARGContentItemType is defined as below.

typedef NS_ENUM (NSInteger, ARGContentItemType) {

ARGContentItemTypeSticker,

ARGContentItemTypeFilter,

ARGContentItemTypeBeauty,

ARGContentItemTypeBulge,

ARGContentItemTypeNum

};

Parameters

  • ARGContentItemType itemType: ARGContentItemType of Contents to Be Set

  • NSString itemFilePath: Saved Path for Contents

  • NSString itemId: uuid of Contents

  • completion: callback result

- (void)setFilterLevel:(float)level

Set Level of An Applied Filter. The range is from 0 to 100. The larger value, the stronger effect.

Parameters

  • float level: Level of Filter (0 ~ 100)

- (void)setDefaultBeauty

Set Level of Face Beautifications to Default.

- (void)setBeautyOn:(BOOL)on

Turn On or Off of a Face Beautification Effect by Setting Boolean Value.

Parameters

  • BOOL on: Set to Turn On or Off. (True : Turn On, False : Turn Off)

- (void)setBeauty:(ARGContentItemBeauty)type value:(float)value

Set a Face Beautification Effect with its Type and Level of Effect.

Parameters

  • ARGContentItemBeauty type: A Type of Face Beautification

  • float value: Level of Effect Intensity

- (float)getBeautyValue:(ARGContentItemBeauty)type

Get the Current Level of a Face Beautification Effect Having ARGContentItemBeauty Type.

Parameters

  • ARGContentItemBeauty type: Type of Face Beautification to Get

Returns

  • float: Level of Face Beautification Value.

- (void)setBeautyValues:(float *_Nonnull)values

Set 16 Types of Face Beautification Levels at Once.

Example Values:

float beautyValue[BEAUTY_TYPE_NUM];

beautyValue[BEAUTY_TYPE_VLINE] = 10.f;

beautyValue[BEAUTY_TYPE_FACE_SLIM] = 90.f;

beautyValue[BEAUTY_TYPE_JAW] = 55.f;

beautyValue[BEAUTY_TYPE_CHIN] = 50.f;

beautyValue[BEAUTY_TYPE_EYE] = 5.f;

beautyValue[BEAUTY_TYPE_EYE_GAP] = -10.f;

beautyValue[BEAUTY_TYPE_NOSE_LINE] = 0.f;

beautyValue[BEAUTY_TYPE_NOSE_SIDE] = 35.f;

beautyValue[BEAUTY_TYPE_NOSE_LENGTH] = 30.f;

beautyValue[BEAUTY_TYPE_MOUTH_SIZE] = -35.f;

beautyValue[BEAUTY_TYPE_EYE_BACK] = 0.f;

beautyValue[BEAUTY_TYPE_EYE_CORNER] = 0.f;

beautyValue[BEAUTY_TYPE_LIP_SIZE] = 0.f;

beautyValue[BEAUTY_TYPE_SKIN_FACE] = 50.f;

beautyValue[BEAUTY_TYPE_SKIN_DARK_CIRCLE] = 0.f;

beautyValue[BEAUTY_TYPE_SKIN_MOUTH_WRINKLE] = 0.f;

To check setting ranges, please refer to 9.2 Apply Face Beautification Effect.

Parameters

  • float *values: A Float Array in Size 16.

- (void)setBulge:(ARGContentItemBulge)bulge

Set a Type of Face Fun Bulge among Available 6 Types. By using ARGContentItemBulgeNONE option, you can disable face fun bulge effect.

ARGContentItemBulge is defined as follows.

typedef NS_ENUM (NSInteger, ARGContentItemBulge) {

ARGContentItemBulgeNONE = -1,

ARGContentItemBulgeFUN1 = 1,

ARGContentItemBulgeFUN2 = 2,

ARGContentItemBulgeFUN3 = 3,

ARGContentItemBulgeFUN4 = 4,

ARGContentItemBulgeFUN5 = 5,

ARGContentItemBulgeFUN6 = 6,

ARGContentItemBulgeNum

};

Parameters

  • ARGContentItemBulge bulge: Face Fun Bulge Type to Be Set

- (void)clear:(ARGContentItemType)itemType

Clear Contents Having ARGContentItemType.

Parameters

  • ARGContentItemType itemType: Please Refer to 12.4.2 setItemWithType Method

14.5 ARGMedia

14.5.1 Overview

Properties

Description

NS_ENUM ARGMediaRatio

Camera Screen Ratio

NS_ENUM ARGMediaMode

Current Media Mode

NS_ENUM ARGMediaVideoBitrate

Video Recording Bitrate

BOOL autoSave

Whether Autosave Mode is On or Not

BOOL recordSound

Whether Including Sound or Not

Method

Description

- setVideoDevice:

Set the Current Camera AVCaptureDevice.

- setVideoConnection:

Set the Current Camera AVCaptureConnection.

- setVideoDeviceOrientation:

Set the Current Camera AVCaptureVideoOrientation.

- setMediaRatio:

Set the Current Camera ARGMediaRatio.

- setMediaMode:

Set the Current Camera ARGMediaMode.

- setVideoBitrate:

Set the Current Camera ARGMediaVideoBitrate.

- takePic:

Save a Picture

- getRecordingOrientation

Return Orientation of Video Recording.

- recordVideoStart:

Start Video Recording.

- recordVideoStop:save:

Stop Video Recording.

14.5.2 Details

NS_ENUM ARGMediaRatio (NSInteger)

Available Camera Screen Ratios Defined as Below.

ARGMediaRatio_16x9: 16:9 Ratio

ARGMediaRatio_4x3: 4:3 Ratio

ARGMediaRatio_1x1: 1:1 Ratio

NS_ENUM ARGMediaMode (NSInteger)

Available Media Mode Defined as Below.

ARGMediaModePhoto: Photo Mode

ARGMediaModeVideo: Video Mode

NS_ENUM ARGMediaVideoBitrate (NSInteger)

Available Video Recording Bitrate Defined as Below.

ARGMediaVideoBitrate_4M: 4000000 Bitrate

ARGMediaVideoBitrate_2M: 2000000 Bitrate

ARGMediaVideoBitrate_1M: 1000000 Bitrate

BOOL autoSave

To Determine whether Saving a Photo or Video Automatically. Default value is NO. When the autoSave variable is set as YES, it automatically saves contents.

BOOL recordSound

To Determine whether It Should Record Sound or Not. Default value is YES. This variable should be set before start recording.

- (void)setVideoDevice:(AVCaptureDevice *)device

This function should be called before using media functionalities. It sets AVCaptureDevice of the current camera.

Parameters

  • AVCaptureDevice device: AVCaptureDevice of of the Current Camera, Rendering CMSampleBufferRef to ARGSession

- (void)setVideoConnection:(AVCaptureConnection *)connection

This function should be called before using media functionalities. It sets AVCaptureConnection of the current camera.

Parameters

  • AVCaptureConnection connection:

    • AVCaptureConnection of the Current Camera, Rendering CMSampleBufferRef to ARGSession.

- (void)setVideoDeviceOrientation:(AVCaptureVideoOrientation)videoOrientation

This function should be called before using media functionalities. It sets AVCaptureVideoOrientation of the current camera.

Parameters

  • AVCaptureVideoOrientation videoOrientation: AVCaptureVideoOrientation of the Current Camera, Rendering CMSampleBufferRef to ARGSession.

- (void)setMediaRatio:(ARGMediaRatio)mediaRatio

This function should be called before using media functionalities. It sets ARGMediaRatio of the current camera.

Parameters

  • ARGMediaRatio mediaRatio: Screen Size Ratio of the Current Camera, Rendering CMSampleBufferRef to ARGSession.

- (void)setMediaMode:(ARGMediaMode)mediaMode

This function should be called before using media functionalities. It sets ARGMediaMode of the current camera.

Parameters

  • ARGMediaMode mediaMode: To Determine Photo or Video Mode

- (void)setVideoBitrate:(ARGMediaVideoBitrate)videoBitrate

This function should be called before using media functionalities. It sets ARGMediaVideoBitrate of the current camera. Higher bitrate records higher resolution.

Parameters

  • ARGMediaVideoBitrate videoBitrate: Video Bitrate to Be Recorded

- (void)takePic:(void (^ __nullable)(UIImage * _Nullable image))completion

Called when Taking a Photo. The function returns a captured image in UIImage format. When Contents are set, those rendered Contents are captured together.

Parameters

  • void (^ __nullable)(UIImage * _Nullable image)completion: Returns the Result of an Image in UIImage Format.

- (AVCaptureVideoOrientation)getRecordingOrientation

Return Device Orientation of a Camera during Video Recording.

Returns

  • AVCaptureVideoOrientation: Camera Device Orientation

- (void)recordVideoStart:(void (^ __nullable)(CGFloat recTime))completion

Called when Starting Video Recording. The function returns recTime included block continuously.

Parameters

  • void (^ __nullable)(CGFloat recTime)completion: Returns Recorded Time in recTime Variable Continuously.

- (void)recordVideoStop:(void (^ __nullable)(NSDictionary * __nonnull videoInfo))stopBlock

save:(void (^ __nullable)(NSDictionary * __nonnull videoInfo))saveBlock

Used when Stop Video Recording. The function returns saveBlock, which is converted from stopBlock after recording stopped.

Parameters

  • void (^ __nullable)(NSDictionary * __nonnull videoInfo)stopBlock: Returned when Video Recording Stopped. When recording sound is on, video and audio are stored separately in videoInfo.

  • void (^ __nullable)(NSDictionary * __nonnull videoInfo)saveBlock: Returned when Video and Audio Are Merged. The merged content is stored in videoInfo.

14.6 ARGConfig

14.6.1 Overview

Method

Description

- initWithApiURL:apiKey:secretKey:authKey:

Initialize ARGConfig with API information.

14.6.2 Details

- (id)initWithApiURL:(NSString *)apiURL apiKey:(NSString *)apiKey secretKey:(NSString *)secretKey authKey:(NSString *)authKey

Initialize ARGConfig using API Key information obtained from CMS in argear.io.

Parameters

  • NSString apiURL: URL Containing API Information except apiKey

  • NSString apiKey: API Key

  • NSString secretKey: Secret Key

  • NSString authKey: Authentication Key

Previous13. ARKit Connect APINext1. Quickstart

Last updated