Multi-Grained Spatio-temporal Modeling for Lip-reading

Chenhao Wang

REVIEW 5 major objections 4 minor 27 references

Multi-Grained Spatio-temporal Modeling for Lip-reading

T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-grained lip-reading network with learned fusion and forward attention reports state-of-the-art accuracy on LRW and LRW-1000 benchmarks.

desk verdict A modest fusion-mask architecture with a small LRW gain, undermined by a baseline reproduction gap and a SOTA claim its own table contradicts. read the letter →

arxiv 1908.11618 v2 pith:RM2QNJLR submitted 2019-08-30 cs.CV eess.AS

Chenhao Wang This is my paper · ORCID

classification cs.CVeess.AS

keywords lip-readingvisualspeechrecognitionmulti-grainedspatio-temporalmodelingConvLSTMtemporalattentionspatialmaskLRWdatasetLRW-1000

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that word-level lip-reading improves when a network models the speaking process at three spatio-temporal scales at once: per-frame appearance for subtle distinctions between homophemes, short-term motion for how a word is spoken, and whole-sequence aggregation for robustness to speaker, pose, and lighting variation. To combine the first two scales, it learns a spatial attention mask that decides, at each position, whether to trust the frame-level 2D branch or the motion-sensitive 3D branch. To aggregate over the full sequence, it uses a two-layer bidirectional ConvLSTM with an attention gate on its forward input, which lets the model weight frames by importance. The paper reports 83.34% word accuracy on LRW and 36.91% on LRW-1000, and credits the multi-grained design for the improvement over its reproduced baselines.

What carries the argument

The load-bearing object is the multi-grained spatio-temporal network. It runs two branches in parallel: a 34-layer 2D ResNet that produces per-frame, fine-grained appearance features $S$, and a 52-layer 3D DenseNet that produces short-term, motion-sensitive features $T$. A $1\times1\times1$ convolutional layer learns a spatial mask $M=\sigma(WT)$, and the branches are fused as $F = T\odot M + S\odot (1-M)$, so each spatial location can choose between appearance and motion evidence. The fused features feed a two-layer bidirectional ConvLSTM whose forward direction is gated by an input attention unit $a_t=\sigma(W_{Xa}X_{f;t}+W_{ha}h_{f;t-1})$; this selects important frames while preserving spatial structure. Together these components are what the paper credits for its accuracy figures.

What would settle it

Re-run the proposed model and the original 3D+2D baseline on LRW-1000 with the exact protocol of [23], including the fully-connected layers and three-stage training, and check whether the proposed model exceeds the published 38.19% accuracy; if it does not, the architecture's claimed advantage over the state of the art is not established.

Watch

Extended reading notes

Core claim

The central claim is that a multi-grained spatio-temporal network, rather than any single-scale video model, is the right architecture for word-level lip-reading. The paper argues that homophemes require frame-level appearance detail, that most word distinctions live in short-term motion captured by 3D convolution, and that whole-sequence aggregation with a bidirectional ConvLSTM supplies robustness to speaker and appearance variation. It reports that its architecture, a 2D ResNet-34 branch plus a 3D DenseNet-52 branch fused by a learned spatial mask and followed by a forward-input-attention Bi-ConvLSTM, achieves 83.34% accuracy on LRW and 36.91% on LRW-1000. These numbers are presented as state of the art, with the LRW result exceeding the previous 83.0% mark, and as evidence that multi-grained modeling is what makes the difference.

Load-bearing premise

The load-bearing premise is that the baseline systems used for comparison are trained as strongly as the original published systems; the paper's own reproduction of the 3D+2D baseline on LRW-1000 is 4.41 percentage points below the published score, which it attributes to omitted fully-connected layers and missing three-stage training, so if the same under-training affects the other baselines, the claimed gains could be an artifact of weaker comparison systems.

Editorial extensions

If this is right

If the multi-grained design works as claimed, word-level lip-reading systems can be built by pairing any strong 2D face encoder with a 3D motion encoder and letting the learned mask decide how to combine them at each spatial location, instead of hand-tuning the fusion.
A forward attention gate on a Bi-ConvLSTM gives the model a way to down-weight frames that contain surrounding context rather than the target word; on LRW clips, which include such context, this should reduce word error rate.
Because ConvLSTM keeps spatial layout through the recurrence, the network can average predictions over spatial positions at the output, a simple decision rule the paper uses for its final accuracy.
The architecture separates feature extraction from temporal aggregation, so stronger visual backbones can be substituted into the two branches without altering the fusion mask or the recurrent stage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A fair test of the method's value would be to pit it against the original 3D+2D baseline of [23] under identical three-stage training with fully-connected layers; the paper's own reproduction of that baseline is 4.41 points below the published number, so the comparison could change.
The learned mask could be inspected beyond the lip region: if it allocates weight to teeth, tongue, or jaw, that would indicate the model exploits articulator cues beyond the cropped lips; if it does not, the mask's job may be mostly to suppress background.
The attention gate could be ablated against a fixed gate of the same capacity; if accuracy barely changes, the observed gain may come from added parameters or regularization rather than from frame selection, a distinction the paper does not directly test.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a modest architecture paper with a plausible idea and a small gain on LRW, but the headline claim is contradicted by the paper's own table on LRW-1000, and the baseline reproduction gap makes the reported advantages hard to trust.

What's actually new: the two-branch front-end with a learned position-wise fusion mask (Eq. 1) is a small but real departure from simple concatenation, and the forward input attention on the Bi-ConvLSTM is a sensible addition. The authors do run control experiments comparing single-branch, two-branch with Bi-GRU, two-branch with Bi-ConvLSTM, and the full model, so the ablation story is at least present.

The soft spots are concentrated in Table 2. First, the SOTA claim: on LRW-1000, the paper's own Table 2(b) lists 3D+2D [23] at 38.19%, while the proposed model gets 36.91%. The conclusion says "state-of-the-art performance on two challenging lip-reading datasets" — that's simply false for LRW-1000 as written.

Second, and more worrying, the baseline reproduction issue. The authors reproduce 3D+2D at 33.78% versus the published 38.19%, a 4.41-point gap they attribute to missing fully-connected layers and missing three-stage training (Sec. 4.4). That means their improvement over the reproduced baseline is measured against a version of the prior method that is evidently under-trained. Since the reported gains are tiny — 0.34% over Stafylakis17 on LRW, and negative on LRW-1000 — even a small amount of unfairness in the other reproduced baselines could flip the conclusion. The paper doesn't provide enough implementation detail to rule that out.

Also missing: code, error bars, and repeated-run statistics. For a paper whose main positive result is a 0.3-point improvement, single-run numbers are not enough to separate signal from noise.

Who is this for? Researchers working on lip-reading or video sequence modeling might pick up the fusion mask and the attention-augmented ConvLSTM as design options. The architecture is coherent and the writing is honest about the baseline gap, but the empirical case is thin. As an editor, I'd send it to a serious referee because the architecture question is legitimate and the flaws are fixable with better baselines and a re-scoped claim. It should not be rejected simply for being incremental, but the SOTA claim needs to go or be heavily qualified.

Recommendation: engage if you work in this area, primarily for the fusion mechanism. For peer review, yes — give it a careful referee, but expect the comparison to need substantial work.

Referee Report

5 major / 4 minor

Summary. This paper proposes a multi-grained spatio-temporal network for word-level lip-reading. The front-end uses parallel 2D ResNet-34 and 3D DenseNet-52 branches to extract fine- and medium-grained features, which are fused by a learned spatial attention mask. A two-layer bidirectional ConvLSTM with forward input attention then aggregates long-range spatio-temporal information. On LRW the method achieves 83.34% and on LRW-1000 36.91%; the authors claim state-of-the-art performance on both benchmarks. The paper also reports ablations comparing two-branch Bi-GRU, two-branch Bi-ConvLSTM, and the full proposed model.

Significance. If the results were fully supported, the paper would make a meaningful contribution: it identifies a plausible limitation of cascaded 3D+2D architectures, introduces an adaptive fusion mechanism, and is apparently the first to apply ConvLSTM to lip-reading. The ablation structure is reasonable, and the disclosure that the reproduced 3D+2D baseline is weaker than the published number is candid. However, the central empirical claim is not currently supported: the paper's own Table 2(b) lists a published method (3D+2D, 38.19%) that beats the proposed model on LRW-1000 by 1.28 points, and the only comparison against that method uses an under-trained reproduction. The margins over the next-best published result on LRW are also very small (0.34%), with no error bars. The paper therefore needs additional experiments and a careful reframing of its claims.

major comments (5)

[Section 4.4 / Table 2(b) / Conclusion] The conclusion claims state-of-the-art performance on two benchmarks, but Table 2(b) lists 3D+2D [23] at 38.19% on LRW-1000, which is 1.28 points above the proposed model's 36.91%. The state-of-the-art claim for LRW-1000 is therefore contradicted by the paper's own reported numbers; please retract or substantially qualify the claim and provide a direct comparison with the published 3D+2D system.
[Section 4.4 / Table 2(b)] The reproduced 3D+2D baseline reaches 33.78% on LRW-1000 versus the published 38.19%, and the authors attribute this 4.41-point gap to omitted fully-connected layers and the missing three-stage training. Because the proposed model's advantage over this reproduction is only 3.13 points, the experiment does not establish that the proposed architecture is superior to a properly trained 3D+2D baseline; a comparison using the full training protocol of [23] is required before the improvement claim can be assessed.
[Section 4.3 / Table 1] The text states that bidirectional ConvLSTM modules 'significantly improve' performance over two-branch Bi-GRU, but Table 1 shows only a 0.17% gain on LRW (82.98% to 83.15%) and a decrease on LRW-1000 (36.48% to 36.12%). This claim is not supported by the reported numbers, and the paper should avoid significance language unless error bars or statistical tests are provided.
[Table 1 / Table 2(b)] The baseline labels are internally inconsistent: Table 1 attributes 38.19% on LRW-1000 to 'ResNet-34 + Bi-GRU [23]' and 34.76% to 'DenseNet-3D + Bi-GRU [23]', while Table 2(b) attributes 38.19% to 3D+2D [23] and 34.76% to D3D [23]. The reader cannot tell which numbers are cited from [23] and which are the author's own reproductions; please correct the table and clarify the provenance of every row.
[Section 4.3 / Table 1 / Table 2] All reported accuracies are single-run point estimates with no error bars, standard deviations, or significance tests. The key margins are small (0.34% over Stafylakis17 on LRW and 0.19% for the forward-attention ablation on LRW), so the reported differences may be within run-to-run variation; please report multiple seeds or an uncertainty estimate and adjust the conclusions accordingly.

minor comments (4)

[Section 3.3.1] Please specify how the forward and backward hidden states of the Bi-ConvLSTM are combined at each time step and how the final prediction is obtained; the current text only says framewise predictions are averaged.
[Section 4.2] Please report the hidden-state channel count of the Bi-ConvLSTM, the number of training epochs, batch size, and the exact two-stage training schedule; these details are necessary for reproducibility.
[Abstract / Section 4.4] The phrase 'coarse-gained' appears in the abstract and should be 'coarse-grained'; the manuscript also contains several ungrammatical sentences (e.g., the sentence about the reproduced 3D+2D in Section 4.4), and a careful proofread is needed.
[Figure 1] The tensor dimensions and the configuration of the two-branch front-end are difficult to read in Figure 1; please enlarge or redraw the figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity.

full rationale

Walked the paper's claimed derivation chain: the central result is an architecture (two-branch 2D/3D front-end with a learned fusion mask and a Bi-ConvLSTM with forward input attention) evaluated by classification accuracy on the public LRW and LRW-1000 benchmarks (Sec. 4.3, Table 1). There is no step in which a predicted quantity is defined in terms of the fitted target, no parameter fitted to a subset and then reported as a prediction for a closely related quantity, and no load-bearing uniqueness theorem imported from the authors' prior work. The learned fusion mask (Eq. 1) and attention weights (Eqs. 3–4) are trained and then measured; their effect is empirically compared against control architectures, so the evaluation is self-contained with respect to the external benchmarks. The weak reproduced 3D+2D baseline (33.78% vs. 38.19% in [23], Sec. 4.4) is a legitimate comparison-fairness concern, and the paper's own Table 2(b) shows the proposed model (36.91%) does not beat the originally reported 3D+2D (38.19%) on LRW-1000, but these are correctness and benchmark-fairness issues rather than circularity. The paper explicitly attributes the reproduction gap to omitted fully-connected layers and missing three-stage training (Sec. 4.4), which is an implementation-fidelity note, not an input-dependent derivation. No substitution of a fitted constant for a prediction, no definitional equivalence, and no self-citation chain forces the claimed outcome. Verdict: no significant circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its empirical claims rest on public datasets, a two-branch fusion design, and a training schedule. The main unaccounted element is the fully trained model itself, which is not released.

free parameters (2)

Learned network weights and fusion mask W = not released
The accuracy claims are empirical fits to the LRW and LRW-1000 training sets; the trained weights are not released, so the central numbers cannot be independently reproduced.
Architecture hyperparameters = not justified
Layer counts (ResNet-34, Dense3D-52), ConvLSTM kernel size 3x3, fusion kernel 1x1x1, input size 88, and maximum frame count 30 are hand-selected with no ablations or sensitivity analysis.

assumptions (3)

domain assumption LRW and LRW-1000 labels and bounding-box annotations are correct, and the standard train/validation/test splits are used.
The paper relies on these public benchmarks without independent verification of label quality or split protocol.
ad hoc to paper Two-stage training, pretraining each branch to convergence and then fine-tuning end-to-end, is a fair protocol for comparing with prior end-to-end methods.
The authors use this recipe for their models but note that omitted fully-connected layers and missing three-stage training lower their LRW-1000 reproduction of [23], undermining fair comparison.
domain assumption The learned fusion mask T * mask + S * (1 - mask) preserves more discriminative information than simple concatenation or cascading.
This is the core architectural premise; the paper provides only a mask visualization and a small accuracy delta, not a controlled comparison against alternative fusion strategies.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Grained Spatio-temporal Modeling for Lip-reading." pith.science (2026). https://pith.science/paper/RM2QNJLR

@misc{pith2026190811618,
  author       = {Pith},
  title        = {Pith review of: Multi-Grained Spatio-temporal Modeling for Lip-reading},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RM2QNJLR}},
  note         = {Machine review of arXiv:1908.11618}
}

read the original abstract

Lip-reading aims to recognize speech content from videos via visual analysis of speakers' lip movements. This is a challenging task due to the existence of homophemes-words which involve identical or highly similar lip movements, as well as diverse lip appearances and motion patterns among the speakers. To address these challenges, we propose a novel lip-reading model which captures not only the nuance between words but also styles of different speakers, by a multi-grained spatio-temporal modeling of the speaking process. Specifically, we first extract both frame-level fine-grained features and short-term medium-grained features by the visual front-end, which are then combined to obtain discriminative representations for words with similar phonemes. Next, a bidirectional ConvLSTM augmented with temporal attention aggregates spatio-temporal information in the entire input sequence, which is expected to be able to capture the coarse-gained patterns of each word and robust to various conditions in speaker identity, lighting conditions, and so on. By making full use of the information from different levels in a unified framework, the model is not only able to distinguish words with similar pronunciations, but also becomes robust to appearance changes. We evaluate our method on two challenging word-level lip-reading benchmarks and show the effectiveness of the proposed method, which also demonstrate the above claims.

Figures

Figures reproduced from arXiv: 1908.11618 by the authors.

**Figure 2.** (a) The forward input attention augmented Bi-ConvLSTM. The attention [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The attention mask automatically adjusts the position-specific fusion weights and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 24 canonical work pages

[23]

Lrw-1000: A naturally-distributed large-scale benchmark for lip reading in the wild

Shuang Yang, Yuanhang Zhang, Dalu Feng, Mingmin Yang, Chenhao Wang, Jingyun Xiao, Keyu Long, Shiguang Shan, and Xilin Chen. Lrw-1000: A naturally-distributed large-scale benchmark for lip reading in the wild. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) , pages 1–8. IEEE, 2019

work page 2019
[1]

Improved speaker independent lipreading using speaker adaptive training and deep neural networks

Ibrahim Almajai, Stephen Cox, Richard Harvey, and Yuxuan Lan. Improved speaker independent lipreading using speaker adaptive training and deep neural networks. In 10 W ANG: MULTI-GRAINED SPA TIO-TEMPORAL MODELING FOR LIP-READING IEEE International Conference on Acoustics , 2016

work page 2016
[2]

Lip- net: End-to-end sentence-level lipreading

Yannis M Assael, Brendan Shillingford, Shimon Whiteson, and Nando de Freitas. Lip- net: End-to-end sentence-level lipreading. arXiv preprint arXiv:1611.01599, 2016

arXiv 2016
[3]

The natural statistics of audiovisual speech

Chandramouli Chandrasekaran, Andrea Trubanova, Sébastien Stillittano, Alice Caplier, and Asif A Ghazanfar. The natural statistics of audiovisual speech. PLoS computational biology, 5(7):e1000436, 2009

work page 2009
[4]

Lipreading from color video

Greg I Chiou and Jenq-Neng Hwang. Lipreading from color video. IEEE Transactions on Image Processing, 6(8):1192–1195, 1997

work page 1997
[5]

Lip reading in the wild

Joon Son Chung and Andrew Zisserman. Lip reading in the wild. In Asian Conference on Computer Vision, pages 87–103. Springer, 2016

work page 2016
[6]

Learning to lip read words by watching videos

Joon Son Chung and Andrew Zisserman. Learning to lip read words by watching videos. Computer Vision and Image Understanding, 2018

work page 2018
[7]

Lip reading sentences in the wild

Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. Lip reading sentences in the wild. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3444–3453. IEEE, 2017

work page 2017

Show all 27 references

[8]

Toward movement-invariant automatic lip-reading and speech recognition

Paul Duchnowski, Martin Hunke, Dietrich Busching, Uwe Meier, and Alex Waibel. Toward movement-invariant automatic lip-reading and speech recognition. In Acous- tics, Speech, and Signal Processing, 1995. ICASSP-95., 1995 International Conference on, volume 1, pages 109–112. IEEE, 1995

1995
[9]

Long short-term memory

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computa- tion, 9(8):1735–1780, 1997

1997
[10]

Videolstm convolves, attends and ﬂows for action recognition

Zhenyang Li, Kirill Gavrilyuk, Efstratios Gavves, Mihir Jain, and Cees GM Snoek. Videolstm convolves, attends and ﬂows for action recognition. Computer Vision and Image Understanding, 166:41–50, 2018

2018
[11]

Audio-visual speech recognition using deep learning

Kuniaki Noda, Yuki Yamaguchi, Kazuhiro Nakadai, Hiroshi G Okuno, and Tetsuya Ogata. Audio-visual speech recognition using deep learning. Applied Intelligence, 42 (4):722–737, 2015

2015
[12]

End-to-end audiovisual speech recognition

Stavros Petridis, Themos Stafylakis, Pingehuan Ma, Feipeng Cai, Georgios Tz- imiropoulos, and Maja Pantic. End-to-end audiovisual speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 6548–6552. IEEE, 2018

2018
[13]

An image transform ap- proach for hmm based automatic lipreading

Gerasimos Potamianos, Hans Peter Graf, and Eric Cosatto. An image transform ap- proach for hmm based automatic lipreading. In Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on, pages 173–177. IEEE, 1998

1998
[14]

Recent advances in the automatic recognition of audiovisual speech

Gerasimos Potamianos, Chalapathy Neti, Guillaume Gravier, Ashutosh Garg, and An- drew W Senior. Recent advances in the automatic recognition of audiovisual speech. Proceedings of the IEEE, 91(9):1306–1326, 2003. W ANG: MULTI-GRAINED SPA TIO-TEMPORAL MODELING FOR LIP-READING 11

2003
[15]

Lip reading using optical ﬂow and support vector machines

Ayaz A Shaikh, Dinesh K Kumar, Wai C Yau, MZ Che Azemin, and Jayavardhana Gubbi. Lip reading using optical ﬂow and support vector machines. In Image and Signal Processing (CISP), 2010 3rd International Congress on , volume 1, pages 327–

2010
[16]

Convolutional lstm network: A machine learning approach for pre- cipitation nowcasting

Xingjian Shi, Zhourong Chen, Hao Wang, Dit Yan Yeung, Wai Kin Wong, and Wang Chun Woo. Convolutional lstm network: A machine learning approach for pre- cipitation nowcasting. In International Conference on Neural Information Processing Systems, 2015

2015
[17]

Two-stream convolutional networks for ac- tion recognition in videos

Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for ac- tion recognition in videos. InAdvances in neural information processing systems, pages 568–576, 2014

2014
[18]

Combining residual networks with lstms for lipreading

Themos Stafylakis and Georgios Tzimiropoulos. Combining residual networks with lstms for lipreading. arXiv preprint arXiv:1703.04105, 2017

2017 arXiv
[19]

Convolutional long short-term memory networks for recognizing ﬁrst person interactions

Swathikiran Sudhakaran and Oswald Lanz. Convolutional long short-term memory networks for recognizing ﬁrst person interactions. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 2339–2346, 2017

2017
[20]

Improving lip-reading performance for robust audiovisual speech recognition using dnns

Kwanchiva Thangthai, Richard W Harvey, Stephen J Cox, and Barry-John Theobald. Improving lip-reading performance for robust audiovisual speech recognition using dnns. In A VSP, pages 127–131, 2015

2015
[21]

Hu- man action recognition by learning spatio-temporal features with deep neural networks

Lei Wang, Yangyang Xu, Jun Cheng, Haiying Xia, Jianqin Yin, and Jiaji Wu. Hu- man action recognition by learning spatio-temporal features with deep neural networks. IEEE Access, 6:17913–17922, 2018

2018
[22]

Pre- drnn: Recurrent neural networks for predictive learning using spatiotemporal lstms

Yunbo Wang, Mingsheng Long, Jianmin Wang, Zhifeng Gao, and S Yu Philip. Pre- drnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. In Advances in Neural Information Processing Systems , pages 879–888, 2017

2017
[24]

Learning spatiotemporal features using 3dcnn and convolutional lstm for gesture recognition

Liang Zhang, Guangming Zhu, Peiyi Shen, Juan Song, Syed Afaq Shah, and Mo- hammed Bennamoun. Learning spatiotemporal features using 3dcnn and convolutional lstm for gesture recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 3120–3128, 2017

2017
[25]

Adding attentiveness to the neurons in recurrent neural networks

Pengfei Zhang, Jianru Xue, Cuiling Lan, Wenjun Zeng, Zhanning Gao, and Nanning Zheng. Adding attentiveness to the neurons in recurrent neural networks. In Proceed- ings of the European Conference on Computer Vision (ECCV) , pages 135–151, 2018

2018
[26]

Lipreading with local spatiotem- poral descriptors

Guoying Zhao, Mark Barnard, and Matti Pietikainen. Lipreading with local spatiotem- poral descriptors. IEEE Transactions on Multimedia, 11(7):1254–1265, 2009

2009
[27]

Multimodal gesture recog- nition using 3-d convolution and convolutional lstm.IEEE Access, 5:4517–4524, 2017

Guangming Zhu, Liang Zhang, Peiyi Shen, and Juan Song. Multimodal gesture recog- nition using 3-d convolution and convolutional lstm.IEEE Access, 5:4517–4524, 2017

2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [23]

Lrw-1000: A naturally-distributed large-scale benchmark for lip reading in the wild

Shuang Yang, Yuanhang Zhang, Dalu Feng, Mingmin Yang, Chenhao Wang, Jingyun Xiao, Keyu Long, Shiguang Shan, and Xilin Chen. Lrw-1000: A naturally-distributed large-scale benchmark for lip reading in the wild. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) , pages 1–8. IEEE, 2019

work page 2019

[2] [1]

Improved speaker independent lipreading using speaker adaptive training and deep neural networks

Ibrahim Almajai, Stephen Cox, Richard Harvey, and Yuxuan Lan. Improved speaker independent lipreading using speaker adaptive training and deep neural networks. In 10 W ANG: MULTI-GRAINED SPA TIO-TEMPORAL MODELING FOR LIP-READING IEEE International Conference on Acoustics , 2016

work page 2016

[3] [2]

Lip- net: End-to-end sentence-level lipreading

Yannis M Assael, Brendan Shillingford, Shimon Whiteson, and Nando de Freitas. Lip- net: End-to-end sentence-level lipreading. arXiv preprint arXiv:1611.01599, 2016

arXiv 2016

[4] [3]

The natural statistics of audiovisual speech

Chandramouli Chandrasekaran, Andrea Trubanova, Sébastien Stillittano, Alice Caplier, and Asif A Ghazanfar. The natural statistics of audiovisual speech. PLoS computational biology, 5(7):e1000436, 2009

work page 2009

[5] [4]

Lipreading from color video

Greg I Chiou and Jenq-Neng Hwang. Lipreading from color video. IEEE Transactions on Image Processing, 6(8):1192–1195, 1997

work page 1997

[6] [5]

Lip reading in the wild

Joon Son Chung and Andrew Zisserman. Lip reading in the wild. In Asian Conference on Computer Vision, pages 87–103. Springer, 2016

work page 2016

[7] [6]

Learning to lip read words by watching videos

Joon Son Chung and Andrew Zisserman. Learning to lip read words by watching videos. Computer Vision and Image Understanding, 2018

work page 2018

[8] [7]

Lip reading sentences in the wild

Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. Lip reading sentences in the wild. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3444–3453. IEEE, 2017

work page 2017