Automatic recognition of depression based on audio and video: A review

Table 1 Summary of advanced automatic depression estimation methods

Method	Year	Framework	Dataset	Modal	Evaluation Criterion
					MAE	RMSE	Accuracy	F1-score
He and Cao[18]	2018	2DCNN	AVEC2013	A	8.201	10.001	-	-
			AVEC2014	A	8.191	9.999	-	-
SIDD	2023	-	DAIC-WOZ	A	-	-	-	0.601
MSCDR	2022	1DCNN/ RNN	DAIC-WOZ	A	-	-	0.771	0.746
DALF	2023	2DCNN	DAIC-WOZ	A	-	-	-	0.784
STFN	2023	1DCNN	DAIC-WOZ	A	5.38	6.36	0.780	-
Speech Former++	2023	Transformer	DAIC-WOZ	A	-	-	0.733	-
Mao et al[24]	2022	CNN RNN	DAIC-WOZ (5)	A	-	-	-	0.958
LGA-CNN	2020	2DCNN	AVEC2013	V-F	6.59	8.39	-	-
			AVEC2014	V-F	6.51	8.30	-	-
SAN	2022	2DCNN	AVEC2013	V-F	7.02	9.37	-	-
			AVEC2014	V-F	6.59	9.24	-	-
Zhao et al[27]	2023	2DCNN	AVEC2013	V-F	5.97	7.36
			AVEC2014	V-F	5.85	7.23
PRA-Net	2023	2DCNN	AVEC2013	V-F	6.08	7.59	-	-
			AVEC2014	V-F	6.04	7.98	-	-
Yuan and Wang[32]	2019	MLP	Private	V-E	-	-	0.831	-
EnSA	2022	Transformer	Private	V-E	-	-	0.955	-
SATCN	2022	1DCNN	Private	V-B	-	-	0.758	-
Zhao and Wang[35]	2022	Transformer	Private	V-B	-	-	0.729	-
ULCDL	2023	RNN	DAIC-WOZ	A + V	-	-	0.830	0.900
Niu et al[37]	2020	2D/3DCNN	AVEC2013	A + V	6.14	8.16	-	-
			AVEC2014	A + V	5.21	7.03	-	-
Shao et al[38]	2021	RNN/CNN	Private	V + V	-	-	0.854	-
TAMFN	2022	2DCNN	D-Vlog	A + V	-	-	-	0.750
Uddin et al[41]	2022	2DCNN	AVEC2013	A + V	5.38	6.83	-	-
			AVEC2014	A + V	5.03	6.16	-	-

V: Video data; A: Audio data; F: Facial information; B: Body information; MAE: Mean absolute error; RMSE: Root mean squared error; MSCDR: Machine speech chain model for depression recognition; SIDD: Speaker-invariant depression detector; DALF: Depression classification; STFN: Spatial-temporal feature network; LGA-CNN: Local global attention convolutional neural network; SAN: Self-adaptation network; PRA-Net: Part-and-Relation Attention Network; SATCN: Spatial attention-dilated temporal convolutional network; ULCDL: Uncertainty-aware label contrastive and distribution learning; TAMFN: Tme-aware attention-based multi-modal fusion depression detection network; DAIC-WOZ: Distress Analysis Interview Corpus/Wizard-of-Oz set; AVEC: Audio-visual emotion recognition challenges; DCNN: Dilated Convolutional Neural Network; CNN: Convolutional neural network; RNN: Recurrent neural networks.