Expression without images.
Smiles, brow lowering, expressivity, gaze, blinks and head motion from MediaPipe Face Landmarker blendshapes computed on the device. No image or video ever reaches the API.
For research and clinician review. Not a diagnosis.
Facial, step by step.
0:50 with on-screen captions. Every number in it is the engine's output for the published sample.
Read the transcript
01 · Expression without images
Facial behavior, measured from numbers computed on the device. No image is ever sent.
02 · On the device
A face landmarker runs where the camera is and turns each frame into 52 scores.
Only the scores, and optionally head pose, travel to the API.
03 · Expression
A smile counts when it holds above 0.5 for at least 0.3 seconds.
With a cheek raise of 0.2 or more, it is a Duchenne smile.
04 · What comes back
For this 30-second session: 8 smiles a minute, all of them Duchenne, and 18 blinks a minute.
05 · Quality gates
Expression needs 10 seconds of frames with a face; head motion needs pose in most of them.
Blendshapes are close proxies for action units, and the catalog says so for every measure.
06 · One request
One request: frames of scores in, expression, gaze, blinks and head motion out.
The face never leaves the device, and the API stores nothing.
07 · Recap
Expression without images: computed on the device, gated, and compared over time.
Inputs, and only those.
Faces never leave the device. The API receives blendshape scores only and stores nothing.
- 01Per-frame scores for the 52 MediaPipe Face Landmarker blendshapes
- 02Optional head pose (yaw, pitch, roll) per frame
- 03Any frame rate from 1 to 120 frames per second
Published methods, in order.
Each step names the method behind it. The catalog lists the references for every measure.
- 01On the device
The face landmarker runs where the camera is and turns each frame into blendshape scores between 0 and 1.
- 02Numbers only
Only the scores, and optionally head pose, are sent. The API never receives a picture.
- 03Expression
Smile intensity and frequency, Duchenne smiles, brow lowering, lip-corner depressor and dimpler, using ARKit-defined proxies for the classic action units.
- 04Eyes and head
Blink rate, downward gaze, gaze aversion, head motion and its variability.
- 05Gate every value
Expression needs 10 s of frames with a face; head measures need pose in most frames.
The biomarkers, defined.
Every measure links to its catalog entry: definition, method, quality gate, published associations and references.
| Biomarker | Unit | What it measures | Published associations |
|---|---|---|---|
| Expression | |||
| Smile intensitysmileIntensityMean | 0–1 | Mean of mouthSmileLeft and mouthSmileRight across face frames (an AU12 proxy).RDoC: Positive valence systems | Depression↓RAnxiety~Post-traumatic stress↓S+1 |
| Smiles per minutesmileFrequencyPerMin | /min | Smile onsets per minute: smile score of 0.5 or more held for at least 300 ms.RDoC: Positive valence systems | Depression↓R |
| Duchenne smile shareduchenneSmileFraction | 0–1 | Share of smile frames with a cheek raise (cheekSquint 0.2 or more, an AU6 proxy).RDoC: Positive valence systems | Post-traumatic stress↓S |
| Brow loweringbrowLoweringMean | 0–1 | Mean of browDownLeft and browDownRight (an imperfect AU4 proxy).RDoC: Negative valence systems | Depression↑S |
| Lip-corner depressorlipCornerDepressorMean | 0–1 | Mean of mouthFrownLeft and mouthFrownRight (an AU15 proxy). | Depression~R |
| DimplerdimplerMean | 0–1 | Mean of mouthDimpleLeft and mouthDimpleRight (an AU14 proxy).RDoC: Social processes | Depression↑R |
| Facial expressivityexpressivityMean | 0–1 | Mean activation of the expression blendshapes (blinks and gaze excluded).RDoC: Positive valence systems | Depression↓RSchizophrenia↓R |
| Expression variabilityexpressivityVariability | 0–1 | Mean temporal standard deviation of the expression blendshapes.RDoC: Positive valence systems | Depression↓RSchizophrenia↓R |
| Eyes and head | |||
| Blink rateblinkRatePerMin | /min | Blink onsets per minute (eyeBlink 0.5 or more, 100 ms refractory). | Depression~Schizophrenia↑MA |
| Downward gazedownwardGazeFraction | 0–1 | Share of frames looking down (eyeLookDown 0.5 or more).RDoC: Social processes | Depression↑RAnxiety↑RPost-traumatic stress↑S |
| Gaze aversiongazeAversionFraction | 0–1 | Share of frames with gaze away from the camera axis in any direction.RDoC: Social processes | Depression↑RAnxiety↑R |
| Head motionheadMotionDegPerSec | °/s | Mean angular speed of the head across consecutive frames.RDoC: Sensorimotor systems | Depression↓R |
| Head pitch variabilityheadPitchSdDeg | ° | Standard deviation of head pitch.RDoC: Sensorimotor systems | Suicide risk↓S |
| Head yaw variabilityheadYawSdDeg | ° | Standard deviation of head yaw. | Descriptive |
Associations are group-level findings from the literature (↑ higher, ↓ lower; MA meta-analysis, R replicated, S single study), never a diagnosis. Compare a person with themselves over time.
Null, never a guess.
When the data cannot support a measure, the value is null and the response says why in quality.flags.
- At least 10 s of frames with a detected face.
- Head motion needs head pose in at least 80% of face frames.
- Blendshapes are proxies for action units, not FACS coding.
One request. Try it now.
Each example downloads the published sample and sends it with the public sandbox key. The sandbox takes synthetic data only.
# The published sample request (synthetic, no real person's data)
curl -s https://www.medera.info/api/v1/samples/facial -o facial.json
curl -s https://www.medera.info/api/v1/facial/analyze \
-H "Authorization: Bearer mk_sandbox_medera_public" \
-H "Content-Type: application/json" \
--data @facial.jsonSends the synthetic sample to the live sandbox with the public key and shows the full response. Nothing personal is sent and nothing is stored.
{
"object": "facial.analysis",
"api_version": "v1",
"mode": "sandbox",
"engine": {
"name": "medera-inference",
"version": "1.0.0",
"vocal": "BH-ASR voice layer 1.2.0 (TypeScript port, number-for-number with the Python reference)"
},
"intended_use": "Measured biomarkers and within-person change for research and clinician review. Not a diagnosis and not a medical device.",
"biomarkers": {
"smileIntensityMean": 0.2025,
"smileFrequencyPerMin": 8,
"duchenneSmileFraction": 1,
"browLoweringMean": 0.1053,
"lipCornerDepressorMean": 0.0346,
"dimplerMean": 0.0352,
"expressivityMean": 0.0532,
"expressivityVariability": 0.0264,
"blinkRatePerMin": 18,
"downwardGazeFraction": 0.1822,
"gazeAversionFraction": 0.1822,
"headMotionDegPerSec": 1.576,
"headPitchSdDeg": 1.332,
"headYawSdDeg": 2.838
},
"quality": {
"frames": 450,
"face_frames": 450,
"face_s": 30,
"pose_frames": 450,
"flags": []
}
}id and a created timestamp. Full referenceFacial, answered.
Still wondering about something?
A real person will answer, usually the same day.
Privacy and portability. Blendshapes are computed on the device by an open model, so the person's face never travels and any platform can send the same numbers.
No. They are close proxies (for example mouthSmile for AU12 and cheekSquint for AU6), and the catalog says so for every measure.
Better together, compared over time.
Send this modality with the others in one call to /v1/analyze, or pass the biomarkers to the inference engine.
Start measuring.
Sandbox today. Production under agreement.
Or make a first request with the sandbox key.