Facial biomarkers

Expression without images.

Smiles, brow lowering, expressivity, gaze, blinks and head motion from MediaPipe Face Landmarker blendshapes computed on the device. No image or video ever reaches the API.

For research and clinician review. Not a diagnosis.

medera · /v1/facial/analyze
sample
On the deviceframe 151 · 15 fpsBlendshape scores, this framemouthSmileLeft0.72mouthSmileRight0.72cheekSquintLeft0.34cheekSquintRight0.34eyeBlinkLeft0.92eyeBlinkRight0.92eyeLookDownLeft0.12browDownLeft0.13white tick: the engine's thresholdSmile intensity0.5smile · Duchennesmile · DuchenneCameraOn-device landmarker52 scores per frameMedera APINo image or video leaves the device. The API receives the scores only.Returned for the 30 s sessionSmiles per minute8Duchenne smile share1Blink rate18/minDownward gaze0.1822
Explained

Facial, step by step.

0:50 with on-screen captions. Every number in it is the engine's output for the published sample.

Read the transcript
  1. 01 · Expression without images

    Facial behavior, measured from numbers computed on the device. No image is ever sent.

  2. 02 · On the device

    A face landmarker runs where the camera is and turns each frame into 52 scores.

    Only the scores, and optionally head pose, travel to the API.

  3. 03 · Expression

    A smile counts when it holds above 0.5 for at least 0.3 seconds.

    With a cheek raise of 0.2 or more, it is a Duchenne smile.

  4. 04 · What comes back

    For this 30-second session: 8 smiles a minute, all of them Duchenne, and 18 blinks a minute.

  5. 05 · Quality gates

    Expression needs 10 seconds of frames with a face; head motion needs pose in most of them.

    Blendshapes are close proxies for action units, and the catalog says so for every measure.

  6. 06 · One request

    One request: frames of scores in, expression, gaze, blinks and head motion out.

    The face never leaves the device, and the API stores nothing.

  7. 07 · Recap

    Expression without images: computed on the device, gated, and compared over time.

What goes in

Inputs, and only those.

Faces never leave the device. The API receives blendshape scores only and stores nothing.

  • 01Per-frame scores for the 52 MediaPipe Face Landmarker blendshapes
  • 02Optional head pose (yaw, pitch, roll) per frame
  • 03Any frame rate from 1 to 120 frames per second
How it is measured

Published methods, in order.

Each step names the method behind it. The catalog lists the references for every measure.

  1. 01On the device

    The face landmarker runs where the camera is and turns each frame into blendshape scores between 0 and 1.

  2. 02Numbers only

    Only the scores, and optionally head pose, are sent. The API never receives a picture.

  3. 03Expression

    Smile intensity and frequency, Duchenne smiles, brow lowering, lip-corner depressor and dimpler, using ARKit-defined proxies for the classic action units.

  4. 04Eyes and head

    Blink rate, downward gaze, gaze aversion, head motion and its variability.

  5. 05Gate every value

    Expression needs 10 s of frames with a face; head measures need pose in most frames.

What comes out

The biomarkers, defined.

Every measure links to its catalog entry: definition, method, quality gate, published associations and references.

BiomarkerUnitWhat it measuresPublished associations
Expression
Smile intensitysmileIntensityMean0–1Mean of mouthSmileLeft and mouthSmileRight across face frames (an AU12 proxy).RDoC: Positive valence systemsDepression↓RAnxiety~Post-traumatic stress↓S+1
Smiles per minutesmileFrequencyPerMin/minSmile onsets per minute: smile score of 0.5 or more held for at least 300 ms.RDoC: Positive valence systemsDepression↓R
Duchenne smile shareduchenneSmileFraction0–1Share of smile frames with a cheek raise (cheekSquint 0.2 or more, an AU6 proxy).RDoC: Positive valence systemsPost-traumatic stress↓S
Brow loweringbrowLoweringMean0–1Mean of browDownLeft and browDownRight (an imperfect AU4 proxy).RDoC: Negative valence systemsDepression↑S
Lip-corner depressorlipCornerDepressorMean0–1Mean of mouthFrownLeft and mouthFrownRight (an AU15 proxy).Depression~R
DimplerdimplerMean0–1Mean of mouthDimpleLeft and mouthDimpleRight (an AU14 proxy).RDoC: Social processesDepression↑R
Facial expressivityexpressivityMean0–1Mean activation of the expression blendshapes (blinks and gaze excluded).RDoC: Positive valence systemsDepression↓RSchizophrenia↓R
Expression variabilityexpressivityVariability0–1Mean temporal standard deviation of the expression blendshapes.RDoC: Positive valence systemsDepression↓RSchizophrenia↓R
Eyes and head
Blink rateblinkRatePerMin/minBlink onsets per minute (eyeBlink 0.5 or more, 100 ms refractory).Depression~Schizophrenia↑MA
Downward gazedownwardGazeFraction0–1Share of frames looking down (eyeLookDown 0.5 or more).RDoC: Social processesDepression↑RAnxiety↑RPost-traumatic stress↑S
Gaze aversiongazeAversionFraction0–1Share of frames with gaze away from the camera axis in any direction.RDoC: Social processesDepression↑RAnxiety↑R
Head motionheadMotionDegPerSec°/sMean angular speed of the head across consecutive frames.RDoC: Sensorimotor systemsDepression↓R
Head pitch variabilityheadPitchSdDeg°Standard deviation of head pitch.RDoC: Sensorimotor systemsSuicide risk↓S
Head yaw variabilityheadYawSdDeg°Standard deviation of head yaw.Descriptive

Associations are group-level findings from the literature (↑ higher, ↓ lower; MA meta-analysis, R replicated, S single study), never a diagnosis. Compare a person with themselves over time.

Quality gates

Null, never a guess.

When the data cannot support a measure, the value is null and the response says why in quality.flags.

  • At least 10 s of frames with a detected face.
  • Head motion needs head pose in at least 80% of face frames.
  • Blendshapes are proxies for action units, not FACS coding.
POST /v1/facial/analyze

One request. Try it now.

Each example downloads the published sample and sends it with the public sandbox key. The sandbox takes synthetic data only.

# The published sample request (synthetic, no real person's data)
curl -s https://www.medera.info/api/v1/samples/facial -o facial.json

curl -s https://www.medera.info/api/v1/facial/analyze \
  -H "Authorization: Bearer mk_sandbox_medera_public" \
  -H "Content-Type: application/json" \
  --data @facial.json
POST/v1/facial/analyze
Sample request

Sends the synthetic sample to the live sandbox with the public key and shows the full response. Nothing personal is sent and nothing is stored.

Response for the sample · 200
{
  "object": "facial.analysis",
  "api_version": "v1",
  "mode": "sandbox",
  "engine": {
    "name": "medera-inference",
    "version": "1.0.0",
    "vocal": "BH-ASR voice layer 1.2.0 (TypeScript port, number-for-number with the Python reference)"
  },
  "intended_use": "Measured biomarkers and within-person change for research and clinician review. Not a diagnosis and not a medical device.",
  "biomarkers": {
    "smileIntensityMean": 0.2025,
    "smileFrequencyPerMin": 8,
    "duchenneSmileFraction": 1,
    "browLoweringMean": 0.1053,
    "lipCornerDepressorMean": 0.0346,
    "dimplerMean": 0.0352,
    "expressivityMean": 0.0532,
    "expressivityVariability": 0.0264,
    "blinkRatePerMin": 18,
    "downwardGazeFraction": 0.1822,
    "gazeAversionFraction": 0.1822,
    "headMotionDegPerSec": 1.576,
    "headPitchSdDeg": 1.332,
    "headYawSdDeg": 2.838
  },
  "quality": {
    "frames": 450,
    "face_frames": 450,
    "face_s": 30,
    "pose_frames": 450,
    "flags": []
  }
}
Computed by the engine for the published sample. Each call also carries an id and a created timestamp. Full reference
FAQ

Facial, answered.

Still wondering about something?

A real person will answer, usually the same day.

Privacy and portability. Blendshapes are computed on the device by an open model, so the person's face never travels and any platform can send the same numbers.

No. They are close proxies (for example mouthSmile for AU12 and cheekSquint for AU6), and the catalog says so for every measure.

Combine them

Better together, compared over time.

Send this modality with the others in one call to /v1/analyze, or pass the biomarkers to the inference engine.

Start measuring.

Sandbox today. Production under agreement.

Or make a first request with the sandbox key.