publications | Juan Carlos Niebles

2025

ViUniT: Visual Unit Tests for More Robust Visual Programming

Artemis Panagopoulou, Honglu Zhou, Silvio Savarese, Caiming Xiong, Chris Callison-Burch, Mark Yatskar, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, Tennessee. Jun 2025

arXiv Bib Blog Code Website

@inproceedings{Artemis_CVPR_2025,
  title = {{ViUniT}: Visual Unit Tests for More Robust Visual Programming},
  author = {Panagopoulou, Artemis and Zhou, Honglu and Savarese, Silvio and Xiong, Caiming and Callison-Burch, Chris and Yatskar, Mark and Niebles, Juan Carlos},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Nashville, Tennessee},
  year = {2025},
  month = jun,
}

Re-thinking Temporal Search for Long-Form Video Understanding

Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, and 2 more authors

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, Tennessee. Jun 2025

arXiv Bib PDF Code Website Data

@inproceedings{Ye_CVPR_2025,
  author = {Ye, Jinhui and Wang, Zihan and Sun, Haosen and Chandrasegaran, Keshigeyan and Durante, Zane and Eyzaguirre, Cristobal and Bisk, Yonatan and Niebles, Juan Carlos and Adeli, Ehsan and Fei-Fei, Li and Wu, Jiajun and Li, Manling},
  title = {Re-thinking Temporal Search for Long-Form Video Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Nashville, Tennessee},
  year = {2025},
  month = jun,
}

AdaVid: Adaptive Video-Language Pretraining

Chaitanya Patel, Juan Carlos Niebles, and Ehsan Adeli

In CVPR 2025 Workshop on Efficient and On-Device Generation (EDGE). Nashville, Tennessee. Jun 2025

arXiv Bib PDF Website

@inproceedings{Patel_CVPRW_2025,
  author = {Patel, Chaitanya and Niebles, Juan Carlos and Adeli, Ehsan},
  title = {AdaVid: Adaptive Video-Language Pretraining},
  booktitle = {CVPR 2025 Workshop on Efficient and On-Device Generation (EDGE)},
  address = {Nashville, Tennessee},
  year = {2025},
  month = jun,
}

Oral

xLAM: A Family of Large Action Models to Empower AI Agent Systems

Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, and 12 more authors

In The 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025). Albuquerque, New Mexico. Apr 2025

Awarded arXiv Bib Blog Code

Oral

@inproceedings{Zhang_NAACL_2025,
  author = {Zhang, Jianguo and Lan, Tian and Zhu, Ming and Liu, Zuxin and Hoang, Thai and Kokane, Shirley and Yao, Weiran and Tan, Juntao and Prabhakar, Akshara and Chen, Haolin and Liu, Zhiwei and Feng, Yihao and Awalgaonkar, Tulika and Murthy, Rithesh and Hu, Eric and Chen, Zeyuan and Xu, Ran and Niebles, Juan Carlos and Heinecke, Shelby and Wang, Huan and Savarese, Silvio and Xiong, Caiming},
  title = {{xLAM}: A Family of Large Action Models to Empower {AI} Agent Systems},
  booktitle = {The 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025)},
  address = {Albuquerque, New Mexico},
  month = apr,
  year = {2025},
}

VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making

Jake Grigsby, Yuke Zhu, Michael S Ryoo, and Juan Carlos Niebles

In ICLR 2025 Workshop on Scaling Self-Improving Foundation Models without Human Supervision. Singapore. Apr 2025

Bib PDF

@inproceedings{Grigsby_ICLRW_2025,
  title = {{VLM} {Q}-Learning: Aligning Vision-Language Models for Interactive Decision-Making},
  author = {Grigsby, Jake and Zhu, Yuke and Ryoo, Michael S and Niebles, Juan Carlos},
  booktitle = {ICLR 2025 Workshop on Scaling Self-Improving Foundation Models without Human Supervision},
  address = {Singapore},
  month = apr,
  year = {2025},
}

ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs

Shirley Kokane, Ming Zhu, Tulika Manoj Awalgaonkar, Jianguo Zhang, Akshara Prabhakar, Thai Quoc Hoang, Zuxin Liu, Rithesh R N, Liangwei Yang, Weiran Yao, and 7 more authors

In ICLR 2025 Workshop on Building Trust in Language Models and Applications. Singapore. Apr 2025

arXiv Bib PDF

@inproceedings{Kokane_ICLRW_2025,
  title = {ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs},
  author = {Kokane, Shirley and Zhu, Ming and Awalgaonkar, Tulika Manoj and Zhang, Jianguo and Prabhakar, Akshara and Hoang, Thai Quoc and Liu, Zuxin and N, Rithesh R and Yang, Liangwei and Yao, Weiran and Tan, Juntao and Liu, Zhiwei and Wang, Huan and Niebles, Juan Carlos and Heinecke, Shelby and Xiong, Caiming and Savarese, Silvio},
  booktitle = {ICLR 2025 Workshop on Building Trust in Language Models and Applications},
  address = {Singapore},
  year = {2025},
  month = apr,
}

Protecting Images From Manipulations With Deep Optical Signatures

Kevin Arias, Pablo Gomez, Carlos Hinojosa, Juan Carlos Niebles, and Henry Arguello

IEEE Journal of Selected Topics in Signal Processing. Apr 2025

DOI Bib

@article{AriasJSTSP2025,
  author = {Arias, Kevin and Gomez, Pablo and Hinojosa, Carlos and Niebles, Juan Carlos and Arguello, Henry},
  journal = {IEEE Journal of Selected Topics in Signal Processing},
  title = {Protecting Images From Manipulations With Deep Optical Signatures},
  year = {2025},
  month = apr,
  doi = {10.1109/JSTSP.2025.3554136},
}

2024

Streaming Detection of Queried Event Start

Cristobal Eyzaguirre, Eric Tang, Shyamal Buch, Adrien Gaidon, Jiajun Wu, and Juan Carlos Niebles

In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. Vancouver, Canada. Dec 2024

arXiv Bib Blog Code Website Data

@inproceedings{Eyzaguirre_NeurIPS_2024,
  author = {Eyzaguirre, Cristobal and Tang, Eric and Buch, Shyamal and Gaidon, Adrien and Wu, Jiajun and Niebles, Juan Carlos},
  title = {Streaming Detection of Queried Event Start},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track},
  address = {Vancouver, Canada},
  month = dec,
  year = {2024},
}

APIGen: Automated PIpeline for Generating Verifiable and Diverse Function-Calling Datasets

Zuxin Liu, Thai Quoc Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, and 7 more authors

In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. Vancouver, Canada. Dec 2024

arXiv Bib Code Website Data

@inproceedings{Zuxin_NeurIPS_2024,
  author = {Liu, Zuxin and Hoang, Thai Quoc and Zhang, Jianguo and Zhu, Ming and Lan, Tian and Kokane, Shirley and Tan, Juntao and Yao, Weiran and Liu, Zhiwei and Feng, Yihao and Murthy, Rithesh and Yang, Liangwei and Savarese, Silvio and Niebles, Juan Carlos and Wang, Huan and Heinecke, Shelby and Xiong, Caiming},
  title = {{APIGen}: Automated PIpeline for Generating Verifiable and Diverse Function-Calling Datasets},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track},
  address = {Vancouver, Canada},
  month = dec,
  year = {2024},
}

IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos

Yunong Liu, Weiyu Liu, Shubh Khanna, Cristobal Eyzaguirre, Manling Li, Juan Carlos Niebles, Vineeth Ravi, Saumitra Mishra, and Jiajun Wu

In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. Vancouver, Canada. Dec 2024

arXiv Bib Code Website Data

@inproceedings{Yunong_NeurIPS_2024,
  author = {Liu, Yunong and Liu, Weiyu and Khanna, Shubh and Eyzaguirre, Cristobal and Li, Manling and Niebles, Juan Carlos and Ravi, Vineeth and Mishra, Saumitra and Wu, Jiajun},
  title = {{IKEA} Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track},
  address = {Vancouver, Canada},
  month = dec,
  year = {2024},
}

PRACT: Optimizing Principled Reasoning and Acting of LLM Agent

Zhiwei Liu, Weiran Yao, Jianguo Zhang, Zuxin Liu, Liangwei Yang, Rithesh R N, Tian Lan, Ming Zhu, Juntao Tan, Shirley Kokane, and 6 more authors

In Proceedings of the 28th Conference on Computational Natural Language Learning (CoNLL). Miami, FL. Nov 2024

DOI arXiv Bib Website

@inproceedings{Liu_CONLL_2024,
  title = {{PRACT}: Optimizing Principled Reasoning and Acting of {LLM} Agent},
  author = {Liu, Zhiwei and Yao, Weiran and Zhang, Jianguo and Liu, Zuxin and Yang, Liangwei and R N, Rithesh and Lan, Tian and Zhu, Ming and Tan, Juntao and Kokane, Shirley and Hoang, Thai Quoc and Niebles, Juan Carlos and Heinecke, Shelby and Wang, Huan and Savarese, Silvio and Xiong, Caiming},
  booktitle = {Proceedings of the 28th Conference on Computational Natural Language Learning (CoNLL)},
  month = nov,
  year = {2024},
  address = {Miami, FL},
  doi = {10.18653/v1/2024.conll-1.33},
}

X-InstructBLIP: A Framework for Aligning Image, 3D, Audio, Video to LLMs and its Emergent Cross-modal Reasoning

Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Milan, Italy. Oct 2024

DOI arXiv Bib Video Code Poster Website Data

@inproceedings{ArtemisECCV2024,
  author = {Panagopoulou, Artemis and Xue, Le and Yu, Ning and Li, Junnan and Li, Dongxu and Joty, Shafiq and Xu, Ran and Savarese, Silvio and Xiong, Caiming and Niebles, Juan Carlos},
  title = {{X-InstructBLIP}: A Framework for Aligning Image, 3D, Audio, Video to {LLM}s and its Emergent Cross-modal Reasoning},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Milan, Italy},
  month = oct,
  year = {2024},
  doi = {10.1007/978-3-031-72995-9_11},
}

LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer

Ning Yu, Chia-Chih Chen, Zeyuan Chen, Rui Meng, Gang Wu, Paul Josel, Juan Carlos Niebles, Caiming Xiong, and Ran Xu

In European Conference on Computer Vision (ECCV). Milan, Italy. Oct 2024

DOI arXiv Bib Code Website Data

@inproceedings{YuECCV2024,
  author = {Yu, Ning and Chen, Chia-Chih and Chen, Zeyuan and Meng, Rui and Wu, Gang and Josel, Paul and Niebles, Juan Carlos and Xiong, Caiming and Xu, Ran},
  title = {{LayoutDETR}: Detection Transformer Is a Good Multimodal Layout Designer},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Milan, Italy},
  month = oct,
  year = {2024},
  doi = {10.1007/978-3-031-72661-3_10},
}

xGen-VideoSyn: High-fidelity Text-to-Video Synthesis with Compressed Representations

Can Qin, Congying Xia, Krithika Ramakrishnan, Michael Ryoo, Lifu Tu, Yihao Feng, Manli Shu, Honglu Zhou, Anas Awadalla, Jun Wang, and 9 more authors

In AI for Visual Arts Workshop and Challenges (AI4VA), in conjunction with ECCV. Milan, Italy. Sep 2024

arXiv Bib

@inproceedings{CanAI4VA2024,
  title = {{xGen-VideoSyn}: High-fidelity Text-to-Video Synthesis with Compressed Representations},
  author = {Qin, Can and Xia, Congying and Ramakrishnan, Krithika and Ryoo, Michael and Tu, Lifu and Feng, Yihao and Shu, Manli and Zhou, Honglu and Awadalla, Anas and Wang, Jun and Purushwalkam, Senthil and Xue, Le and Zhou, Yingbo and Wang, Huan and Savarese, Silvio and Niebles, Juan Carlos and Chen, Zeyuan and Xiong, Caiming and Xu, Ran},
  booktitle = {AI for Visual Arts Workshop and Challenges (AI4VA), in conjunction with ECCV},
  address = {Milan, Italy},
  month = sep,
  year = {2024},
}

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding

Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and 1 more author

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, Washington. Jun 2024

DOI arXiv Bib Blog Code Data

@inproceedings{Xue_CVPR_2024,
  author = {Xue, Le and Yu, Ning and Zhang, Shu and Panagopoulou, Artemis and Li, Junnan and Martín-Martín, Roberto and Wu, Jiajun and Xiong, Caiming and Xu, Ran and Niebles, Juan Carlos and Savarese, Silvio},
  title = {{ULIP-2}: Towards Scalable Multimodal Pre-training for {3D} Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Seattle, Washington},
  month = jun,
  year = {2024},
  doi = {10.1109/CVPR52733.2024.02558},
}

Hierarchical Point Attention for Indoor 3D Object Detection

Manli Shu, Le Xue, Ning Yu, Roberto Martín-Martín, Caiming Xiong, Tom Goldstein, Juan Carlos Niebles, and Ran Xu

In IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan. May 2024

DOI arXiv Bib

@inproceedings{Shu_ICRA_2024,
  author = {Shu, Manli and Xue, Le and Yu, Ning and Martín-Martín, Roberto and Xiong, Caiming and Goldstein, Tom and Niebles, Juan Carlos and Xu, Ran},
  title = {Hierarchical Point Attention for Indoor 3D Object Detection},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  address = {Yokohama, Japan},
  month = may,
  year = {2024},
  doi = {10.1109/ICRA57147.2024.10610108},
}

Spotlight

Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization

Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh R. N., Zeyuan Chen, Jianguo Zhang, Devansh Arpit, and 5 more authors

In The Twelfth International Conference on Learning Representations (ICLR). Vienna, Austria. May 2024

Awarded arXiv Bib PDF Code

Spotlight

@inproceedings{Yao_ICLR_2024,
  author = {Yao, Weiran and Heinecke, Shelby and Niebles, Juan Carlos and Liu, Zhiwei and Feng, Yihao and Xue, Le and N., Rithesh R. and Chen, Zeyuan and Zhang, Jianguo and Arpit, Devansh and Xu, Ran and Mui, Phil L. and Wang, Huan and Xiong, Caiming and Savarese, Silvio},
  title = {Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization},
  booktitle = {The Twelfth International Conference on Learning Representations (ICLR)},
  address = {Vienna, Austria},
  month = may,
  year = {2024},
}

REX: Rapid Exploration and eXploitation for AI agents

Rithesh Murthy, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Le Xue, Weiran Yao, Yihao Feng, Zeyuan Chen, Akash Gokul, Devansh Arpit, and 5 more authors

In ICLR 2024 Workshop on LLM Agents. Vienna, Austria. May 2024

arXiv Bib

@inproceedings{murthy2023rex,
  title = {{REX}: {R}apid {E}xploration and e{X}ploitation for {AI} agents},
  author = {Murthy, Rithesh and Heinecke, Shelby and Niebles, Juan Carlos and Liu, Zhiwei and Xue, Le and Yao, Weiran and Feng, Yihao and Chen, Zeyuan and Gokul, Akash and Arpit, Devansh and Xu, Ran and Mui, Phil and Wang, Huan and Xiong, Caiming and Savarese, Silvio},
  booktitle = {ICLR 2024 Workshop on LLM Agents},
  address = {Vienna, Austria},
  month = may,
  year = {2024},
}

BOLAA: Benchmarking and orchestrating LLM-augmented autonomous agents

Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, and 5 more authors

In ICLR 2024 Workshop on LLM Agents. Vienna, Austria. May 2024

arXiv Bib Code

@inproceedings{liu2023bolaa,
  title = {{BOLAA}: Benchmarking and orchestrating {LLM}-augmented autonomous agents},
  author = {Liu, Zhiwei and Yao, Weiran and Zhang, Jianguo and Xue, Le and Heinecke, Shelby and Murthy, Rithesh and Feng, Yihao and Chen, Zeyuan and Niebles, Juan Carlos and Arpit, Devansh and Xu, Ran and Mui, Phil and Wang, Huan and Xiong, Caiming and Savarese, Silvio},
  booktitle = {ICLR 2024 Workshop on LLM Agents},
  address = {Vienna, Austria},
  month = may,
  year = {2024},
}

The AgentOhana: Designing Unified Data and Training Pipeline for Effective Agent Learning

Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Juntao Tan, Thai Hoang, Liangwei Yang, Yihao Feng, Zuxin Liu, and 6 more authors

In ICLR 2024 Workshop on LLM Agents. Vienna, Austria. May 2024

arXiv Bib Code

@inproceedings{zhang2024ohana,
  title = {The {AgentOhana}: Designing Unified Data and Training Pipeline for Effective Agent Learning},
  author = {Zhang, Jianguo and Lan, Tian and Murthy, Rithesh and Liu, Zhiwei and Yao, Weiran and Tan, Juntao and Hoang, Thai and Yang, Liangwei and Feng, Yihao and Liu, Zuxin and Awalgaonkar, Tulika and Niebles, Juan Carlos and Savarese, Silvio and Heinecke, Shelby and Wang, Huan and Xiong, Caiming},
  booktitle = {ICLR 2024 Workshop on LLM Agents},
  address = {Vienna, Austria},
  month = may,
  year = {2024},
}

Causal Layering via Conditional Entropy

Itai Feigenbaum, Devansh Arpit, Shelby Heinecke, Juan Carlos Niebles, Weiran Yao, Caiming Xiong, Silvio Savarese, and Huan Wang

In Conference on Causal Learning and Reasoning (CLeaR). Los Angeles, California. Apr 2024

arXiv Bib PDF

@inproceedings{Itai_CLeaR_2024,
  author = {Feigenbaum, Itai and Arpit, Devansh and Heinecke, Shelby and Niebles, Juan Carlos and Yao, Weiran and Xiong, Caiming and Savarese, Silvio and Wang, Huan},
  title = {Causal Layering via Conditional Entropy},
  booktitle = {Conference on Causal Learning and Reasoning (CLeaR)},
  address = {Los Angeles, California},
  month = apr,
  year = {2024},
}

2023

UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild

Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang , Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, and 3 more authors

In Advances in Neural Information Processing Systems (NeurIPS). New Orleans, USA. Dec 2023

arXiv Bib PDF Code Website

@inproceedings{Qin_NeurIPS_2023,
  author = {Qin, Can and Zhang, Shu and Yu, Ning and Feng, Yihao and Yang, Xinyi and Zhou, Yingbo and Wang, Huan and Niebles, Juan Carlos and Xiong, Caiming and Savarese, Silvio and Ermon, Stefano and Fu, Yun and Xu, Ran},
  title = {UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  address = {New Orleans, USA},
  month = dec,
  year = {2023},
}

Temporally Disentangled Representation Learning under Unknown Nonstationarity

Xiangchen Song, Weiran Yao, Yewen Fan, Xinshuai Dong, Guangyi Chen, Juan Carlos Niebles, Eric Xing, and Kun Zhang

In Advances in Neural Information Processing Systems (NeurIPS). New Orleans, USA. Dec 2023

arXiv Bib PDF Code

@inproceedings{Song_NeurIPS_2023,
  author = {Song, Xiangchen and Yao, Weiran and Fan, Yewen and Dong, Xinshuai and Chen, Guangyi and Niebles, Juan Carlos and Xing, Eric and Zhang, Kun},
  title = {Temporally Disentangled Representation Learning under Unknown Nonstationarity},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  address = {New Orleans, USA},
  month = dec,
  year = {2023},
}

Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation

Qichen Fu, Xingyu Liu, Ran Xu, Juan Carlos Niebles, and Kris M. Kitani

In IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France. Oct 2023

DOI arXiv Bib Code Website

@inproceedings{Fu_ICCV_2023,
  author = {Fu, Qichen and Liu, Xingyu and Xu, Ran and Niebles, Juan Carlos and Kitani, Kris M.},
  title = {Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Paris, France},
  month = oct,
  year = {2023},
  doi = {10.1109/ICCV51070.2023.02157},
}

Procedure-Aware Pretraining for Instructional Video Understanding

Honglu Zhou, Roberto Martín-Martín, Mubbasir Kapadia, Silvio Savarese, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada. Jun 2023

DOI arXiv Bib PDF Supp Video Code Poster Slides Website

@inproceedings{Zhou_CVPR_2023,
  author = {Zhou, Honglu and Mart\'{i}n-Mart\'{i}n, Roberto and Kapadia, Mubbasir and Savarese, Silvio and Niebles, Juan Carlos},
  title = {Procedure-Aware Pretraining for Instructional Video Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Vancouver, Canada},
  month = jun,
  year = {2023},
  doi = {10.1109/CVPR52729.2023.01033},
}

ULIP: Learning Unified Representation of Language, Image and Point Cloud for 3D Understanding

Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada. Jun 2023

DOI arXiv Bib PDF Blog Code Website Data

@inproceedings{Xue_CVPR_2023,
  author = {Xue, Le and Gao, Mingfei and Xing, Chen and Martín-Martín, Roberto and Wu, Jiajun and Xiong, Caiming and Xu, Ran and Niebles, Juan Carlos and Savarese, Silvio},
  title = {{ULIP}: Learning Unified Representation of Language, Image and Point Cloud for 3D Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Vancouver, Canada},
  month = jun,
  year = {2023},
  doi = {10.1109/CVPR52729.2023.00120}
}

Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotations

Vibashan VS, Ning Yu, Chen Xing, Can Qin, Mingfei Gao, Juan Carlos Niebles , Vishal M. Patel, and Ran Xu

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada. Jun 2023

DOI arXiv Bib PDF Supp Blog Code Poster Slides Website

@inproceedings{VS_CVPR_2023,
  author = {VS, Vibashan and Yu, Ning and Xing, Chen and Qin, Can and Gao, Mingfei and Niebles, Juan Carlos and Patel, Vishal M. and Xu, Ran},
  title = {Mask-free {OVIS}: Open-Vocabulary Instance Segmentation without Manual Mask Annotations},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Vancouver, Canada},
  month = jun,
  year = {2023},
  doi = {10.1109/CVPR52729.2023.02254}
}

PreViTS: Contrastive Pretraining with Video Tracking Supervision

Brian Chen, Ramprasaath R. Selvaraju, Shih-Fu Chang, Juan Carlos Niebles, and Nikhil Naik

In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Hawaii, USA. Jan 2023

DOI arXiv Bib PDF

@inproceedings{Chen_WACV_2023,
  author = {Chen, Brian and Selvaraju, Ramprasaath R. and Chang, Shih-Fu and Niebles, Juan Carlos and Naik, Nikhil},
  title = {{PreViTS}: Contrastive Pretraining with Video Tracking Supervision},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  address = {Hawaii, USA},
  month = jan,
  year = {2023},
  doi = {10.1109/WACV56688.2023.00161}
}

2022

MOMA-LRG: Language-Refined Graphs for Multi-Object Multi-Actor Activity Parsing

Zelun Luo, Zane Durante, Linden Li, Wanze Xie, Ruochen Liu, Emily Jin , Zhuoyi Huang, Lun Yu Li, Jiajun Wu, Juan Carlos Niebles, and 2 more authors

In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. New Orleans, USA. Dec 2022

Bib PDF Code Website Data

@inproceedings{Luo_NeurIPS_2022,
  author = {Luo, Zelun and Durante, Zane and Li, Linden and Xie, Wanze and Liu, Ruochen and Jin, Emily and Huang, Zhuoyi and Li, Lun Yu and Wu, Jiajun and Niebles, Juan Carlos and Adeli, Ehsan and Fei-Fei, Li},
  title = {{MOMA-LRG}: Language-Refined Graphs for Multi-Object Multi-Actor Activity Parsing},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track},
  address = {New Orleans, USA},
  month = dec,
  year = {2022},
}

Oral

PrivHAR: Recognizing Human Actions From Privacy-preserving Lens

Carlos Hinojosa, Miguel Marquez, Henry Arguello, Ehsan Adeli, Li Fei-Fei, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Tel-Aviv, Israel. Oct 2022

Awarded DOI arXiv Bib PDF Supp Poster Website Talk

Oral

@inproceedings{Hinojosa_ECCV_2022,
  author = {Hinojosa, Carlos and Marquez, Miguel and Arguello, Henry and Adeli, Ehsan and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {{PrivHAR}: Recognizing Human Actions From Privacy-preserving Lens},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Tel-Aviv, Israel},
  month = oct,
  year = {2022},
  doi = {10.1007/978-3-031-19772-7_19},
}

Open Vocabulary Object Detection with Pseudo Bounding-Box Labels

Mingfei Gao, Chen Xing, Juan Carlos Niebles, Junnan Li, Ran Xu, Wenhao Liu, and Caiming Xiong

In European Conference on Computer Vision (ECCV). Tel-Aviv, Israel. Oct 2022

DOI arXiv Bib PDF Blog Code

@inproceedings{Gao_ECCV_2022,
  author = {Gao, Mingfei and Xing, Chen and Niebles, Juan Carlos and Li, Junnan and Xu, Ran and Liu, Wenhao and Xiong, Caiming},
  title = {Open Vocabulary Object Detection with Pseudo Bounding-Box Labels},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Tel-Aviv, Israel},
  month = oct,
  year = {2022},
  doi = {10.1007/978-3-031-20080-9_16},
}

Oral

Revisiting the ‘Video’ in Video-Language Understanding

Shyamal Buch, Cristobal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA. Jun 2022

Awarded DOI arXiv Bib PDF Supp Code Poster Website Talk

Oral

@inproceedings{Buch_CVPR_2022,
  author = {Buch, Shyamal and Eyzaguirre, Cristobal and Gaidon, Adrien and Wu, Jiajun and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Revisiting the `Video' in Video-Language Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {New Orleans, USA},
  month = jun,
  year = {2022},
  doi = {10.1109/CVPR52688.2022.00293},
}

Align and Prompt: Video-and-Language Pre-training with Entity Prompts

Dongxu Li, Junnan Li, Hongdong Li, Juan Carlos Niebles, and Steven CH Hoi

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA. Jun 2022

DOI arXiv Bib PDF Blog Code

@inproceedings{Li_CVPR_2022,
  author = {Li, Dongxu and Li, Junnan and Li, Hongdong and Niebles, Juan Carlos and Hoi, Steven CH},
  title = {Align and Prompt: Video-and-Language Pre-training with Entity Prompts},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {New Orleans, USA},
  month = jun,
  year = {2022},
  doi = {10.1109/CVPR52688.2022.00490},
}

2021

MOMA: Multi-Object Multi-Actor Activity Parsing

Zelun Luo, Wanze Xie, Siddharth Kapoor, Yiyun Liang, Michael Cooper, Juan Carlos Niebles, Ehsan Adeli, and Fei-Fei Li

In Advances in Neural Information Processing Systems (NeurIPS). Online. Dec 2021

Bib PDF Supp Website

@inproceedings{Luo_NeurIPS_2021,
  author = {Luo, Zelun and Xie, Wanze and Kapoor, Siddharth and Liang, Yiyun and Cooper, Michael and Niebles, Juan Carlos and Adeli, Ehsan and Li, Fei-Fei},
  title = {{MOMA}: Multi-Object Multi-Actor Activity Parsing},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  month = dec,
  year = {2021},
  address = {Online},
}

TNT: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification

Andrés Villa, Juan-Manuel Perez-Rua, Vladimir Araujo, Juan Carlos Niebles, Victor Escorcia, and Alvaro Soto

In British Machine Vision Conference (BMVC). Online. Nov 2021

arXiv Bib PDF Supp Talk

@inproceedings{Villa_BMVC_2021,
  author = {Villa, Andr\'{e}s and Perez-Rua, Juan-Manuel and Araujo, Vladimir and Niebles, Juan Carlos and Escorcia, Victor and Soto, Alvaro},
  title = {{TNT}: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification},
  booktitle = {British Machine Vision Conference (BMVC)},
  address = {Online},
  month = nov,
  year = {2021},
}

Oral

Learning Privacy-preserving Optics for Human Pose Estimation

Carlos Hinojosa, Juan Carlos Niebles, and Henry Arguello

In IEEE/CVF International Conference on Computer Vision (ICCV). Online. Oct 2021

Awarded DOI Bib PDF Supp Poster Slides Website Talk

Oral

@inproceedings{Hinojosa_ICCV_2021,
  author = {Hinojosa, Carlos and Niebles, Juan Carlos and Arguello, Henry},
  title = {Learning Privacy-preserving Optics for Human Pose Estimation},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Online},
  month = oct,
  year = {2021},
  doi = {10.1109/ICCV48922.2021.00257},
}

Detecting Human-Object Relationships in Videos

Jingwei Ji, Rishi Desai, and Juan Carlos Niebles

In IEEE/CVF International Conference on Computer Vision (ICCV). Online. Oct 2021

DOI Bib PDF Supp Slides

@inproceedings{Ji_ICCV_2021,
  author = {Ji, Jingwei and Desai, Rishi and Niebles, Juan Carlos},
  title = {Detecting Human-Object Relationships in Videos},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Online},
  month = oct,
  year = {2021},
  doi = {10.1109/ICCV48922.2021.00800},
}

TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild

Vida Adeli, Mahsa Ehsanpour, Ian Reid, Juan Carlos Niebles, Silvio Savarese, Ehsan Adeli, and Hamid Rezatofighi

In IEEE/CVF International Conference on Computer Vision (ICCV). Online. Oct 2021

DOI arXiv Bib PDF

@inproceedings{Adeli_ICCV_2021,
  author = {Adeli, Vida and Ehsanpour, Mahsa and Reid, Ian and Niebles, Juan Carlos and Savarese, Silvio and Adeli, Ehsan and Rezatofighi, Hamid},
  title = {{TRiPOD}: Human Trajectory and Pose Dynamics Forecasting in the Wild},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Online},
  month = oct,
  year = {2021},
  doi = {10.1109/ICCV48922.2021.01314},
}

Quantifying Parkinson’s Disease Motor Severity Under Uncertainty Using MDS-UPDRS Videos

Mandy Lu, Qingyu Zhao, Kathleen L Poston, Edith V Sullivan, Adolf Pfefferbaum, Marian Shahid, Maya Katz, Leila Montaser Kouhsari, Kevin Schulman, Arnold Milstein, and 5 more authors

Medical Image Analysis. Oct 2021

DOI Bib PDF Code

@article{Lu_MedIA_2021,
  author = {Lu, Mandy and Zhao, Qingyu and Poston, Kathleen L and Sullivan, Edith V and Pfefferbaum, Adolf and Shahid, Marian and Katz, Maya and Kouhsari, Leila Montaser and Schulman, Kevin and Milstein, Arnold and Niebles, Juan Carlos and Henderson, Victor W and Fei-Fei, Li and Pohl, Kilian M and Adeli, Ehsan},
  title = {Quantifying Parkinson’s Disease Motor Severity Under Uncertainty Using {MDS-UPDRS} Videos},
  journal = {Medical Image Analysis},
  volume = {73},
  month = oct,
  year = {2021},
  doi = {10.1016/j.media.2021.102179},
}

Home Action Genome: Cooperative Compositional Action Understanding

Nishant Rai, Haofeng Chen, Jingwei Ji, Rishi Desai, Kazuki Kozuka, Shun Ishizaka, Ehsan Adeli, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Online. Jun 2021

DOI arXiv Bib PDF

@inproceedings{Rai_CVPR_2021,
  author = {Rai, Nishant and Chen, Haofeng and Ji, Jingwei and Desai, Rishi and Kozuka, Kazuki and Ishizaka, Shun and Adeli, Ehsan and Niebles, Juan Carlos},
  title = {Home Action Genome: Cooperative Compositional Action Understanding},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Online},
  month = jun,
  year = {2021},
  doi = {10.1109/CVPR46437.2021.01103},
}

Metadata Normalization

Mandy Lu, Qinyu Zhao, Jiequan Zhang, Kilian Pohl, Li Fei-Fei, Juan Carlos Niebles, and Ehsan Adeli

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Online. Jun 2021

DOI arXiv Bib PDF Code Website

@inproceedings{Lu_CVPR_2021,
  author = {Lu, Mandy and Zhao, Qinyu and Zhang, Jiequan and Pohl, Kilian and Fei-Fei, Li and Niebles, Juan Carlos and Adeli, Ehsan},
  title = {Metadata Normalization},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Online},
  month = jun,
  year = {2021},
  doi = {10.1109/CVPR46437.2021.01077},
}

Representation Learning with Statistical Independence to Mitigate Bias

Ehsan Adeli, Qinyu Zhao, Adolf Pfefferbaum, Edith Sullivan, Li Fei-Fei, Juan Carlos Niebles, and Kilian Pohl

In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Online. Jan 2021

DOI arXiv Bib PDF

@inproceedings{Adeli_WACV_2021,
  author = {Adeli, Ehsan and Zhao, Qinyu and Pfefferbaum, Adolf and Sullivan, Edith and Fei-Fei, Li and Niebles, Juan Carlos and Pohl, Kilian},
  title = {Representation Learning with Statistical Independence to Mitigate Bias},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  address = {Online},
  month = jan,
  year = {2021},
  doi = {10.1109/WACV48630.2021.00256}
}

2020

Socially and Contextually Aware Human Motion and Pose Forecasting

Vida Adeli, Ehsan Adeli, Ian Reid, Juan Carlos Niebles, and Hamid Rezatofighi

IEEE Robotics and Automation Letters (RA-L). Oct 2020

DOI arXiv Bib PDF

@article{Adeli_RAL_2020,
  author = {Adeli, Vida and Adeli, Ehsan and Reid, Ian and Niebles, Juan Carlos and Rezatofighi, Hamid},
  title = {Socially and Contextually Aware Human Motion and Pose Forecasting},
  journal = {IEEE Robotics and Automation Letters (RA-L)},
  doi = {10.1109/LRA.2020.3010742},
  volume = {5},
  number = {4},
  month = oct,
  year = {2020},
}

Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson’s Disease Motor Severity

Mandy Lu, Kathleen Poston, Adolf Pfefferbaum, Edith V. Sullivan, Li Fei-Fei, Kilian M. Pohl, Juan Carlos Niebles, and Ehsan Adeli

In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). Online. Oct 2020

DOI arXiv Bib Video Code

@inproceedings{Lu_MICCAI_2020,
  author = {Lu, Mandy and Poston, Kathleen and Pfefferbaum, Adolf and Sullivan, Edith V. and Fei-Fei, Li and Pohl, Kilian M. and Niebles, Juan Carlos and Adeli, Ehsan},
  title = {Vision-based Estimation of {MDS-UPDRS} Gait Scores for Assessing Parkinson’s Disease Motor Severity},
  booktitle = {International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)},
  address = {Online},
  month = oct,
  year = {2020},
  doi = {10.1007/978-3-030-59716-0_61},
}

Explaining VQA predictions using visual grounding and a knowledge base

Felipe Riquelme, Alfredo De Goyeneche, Yundong Zhang, Juan Carlos Niebles, and Alvaro Soto

Image and Vision Computing. Sep 2020

DOI Bib

@article{Riquelme_IVC_2020,
  author = {Riquelme, Felipe and Goyeneche, Alfredo De and Zhang, Yundong and Niebles, Juan Carlos and Soto, Alvaro},
  title = {Explaining VQA predictions using visual grounding and a knowledge base},
  journal = {Image and Vision Computing},
  volume = {101},
  month = sep,
  year = {2020},
  doi = {10.1016/j.imavis.2020.103968},
}

RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition

Linxi Fan, Shyamal Buch, Guanhzi Wang, Ryan Cao, Yuke Zhu, Juan Carlos Niebles, and Li Fei-Fei

In European Conference on Computer Vision (ECCV). Online. Aug 2020

DOI Bib PDF Supp Video Code Website

@inproceedings{Fan_ECCV_2020,
  author = {Fan, Linxi and Buch, Shyamal and Wang, Guanhzi and Cao, Ryan and Zhu, Yuke and Niebles, Juan Carlos and Fei-Fei, Li},
  title = {RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Online},
  month = aug,
  year = {2020},
  doi = {10.1007/978-3-030-58529-7_30},
}

Procedure Planning in Instructional Videos

Chien-Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Online. Aug 2020

DOI arXiv Bib PDF

@inproceedings{Chang_ECCV_2020,
  author = {Chang, Chien-Yi and Huang, De-An and Xu, Danfei and Adeli, Ehsan and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Procedure Planning in Instructional Videos},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Online},
  month = aug,
  year = {2020},
  doi = {10.1007/978-3-030-58621-8_20},
}

Segmenting the Future

Hsu-kuang Chiu, Ehsan Adeli, and Juan Carlos Niebles

IEEE Robotics and Automation Letters (RA-L). Jul 2020

DOI arXiv Bib PDF

@article{Chiu_RAL_2020,
  author = {Chiu, Hsu-kuang and Adeli, Ehsan and Niebles, Juan Carlos},
  title = {Segmenting the Future},
  journal = {IEEE Robotics and Automation Letters (RA-L)},
  volume = {5},
  number = {3},
  month = jul,
  year = {2020},
  doi = {10.1109/LRA.2020.2992184},
}

Action Genome: Actions as Compositions of Spatio-temporal Scene Graphs

Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, Washington, USA. Jun 2020

DOI arXiv Bib PDF Video Code Slides Website Data

@inproceedings{Ji_CVPR_2020,
  author = {Ji, Jingwei and Krishna, Ranjay and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Action Genome: Actions as Compositions of Spatio-temporal Scene Graphs},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Seattle, Washington, USA},
  month = jun,
  year = {2020},
  doi = {10.1109/CVPR42600.2020.01025},
}

Spatio-Temporal Graph for Video Captioning with Knowledge Distillation

Boxiao Pan, Haoye Cai, De-An Huang, Kuan-Hui Lee, Adrien Gaidon, Ehsan Adeli, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, Washington, USA. Jun 2020

DOI arXiv Bib PDF Video Code Poster Slides

@inproceedings{Pan_CVPR_2020,
  author = {Pan, Boxiao and Cai, Haoye and Huang, De-An and Lee, Kuan-Hui and Gaidon, Adrien and Adeli, Ehsan and Niebles, Juan Carlos},
  title = {Spatio-Temporal Graph for Video Captioning with Knowledge Distillation},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Seattle, Washington, USA},
  month = jun,
  year = {2020},
  doi = {10.1109/CVPR42600.2020.01088},
}

Few-Shot Video Classification via Temporal Alignment

Kaidi Cao, Jingwei Ji, Zhangjie Cao, Chien-Yi Chang, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, Washington, USA. Jun 2020

DOI arXiv Bib PDF Video Slides Data

@inproceedings{Cao_CVPR_2020,
  author = {Cao, Kaidi and Ji, Jingwei and Cao, Zhangjie and Chang, Chien-Yi and Niebles, Juan Carlos},
  title = {Few-Shot Video Classification via Temporal Alignment},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Seattle, Washington, USA},
  month = jun,
  year = {2020},
  doi = {10.1109/CVPR42600.2020.01063},
}

Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction

Bingbin Liu, Ehsan Adeli, Zhangjie Cao, Kuan-Hui Lee, Abhijeet Shenoi, Adrien Gaidon, and Juan Carlos Niebles

IEEE Robotics and Automation Letters (RA-L) and IEEE International Conference on Robotics and Automation (ICRA). Paris, France. May 2020

DOI arXiv Bib Video Code Website Data

@article{Liu_RAL_2020,
  author = {Liu, Bingbin and Adeli, Ehsan and Cao, Zhangjie and Lee, Kuan-Hui and Shenoi, Abhijeet and Gaidon, Adrien and Niebles, Juan Carlos},
  title = {Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction},
  journal = {IEEE Robotics and Automation Letters (RA-L) and IEEE International Conference on Robotics and Automation (ICRA)},
  volume = {5},
  number = {2},
  address = {Paris, France},
  month = may,
  year = {2020},
  doi = {10.1109/LRA.2020.2976305},
}

Motion Reasoning for Goal-Based Imitation Learning

De-An Huang, Yu-Wei Chao, Chris Paxton, Xinke Deng, Li Fei-Fei, Juan Carlos Niebles, Animesh Garg, and Dieter Fox

In IEEE International Conference on Robotics and Automation (ICRA). Paris, France. May 2020

DOI arXiv Bib Video

@inproceedings{Huang_ICRA_2020,
  author = {Huang, De-An and Chao, Yu-Wei and Paxton, Chris and Deng, Xinke and Fei-Fei, Li and Niebles, Juan Carlos and Garg, Animesh and Fox, Dieter},
  title = {Motion Reasoning for Goal-Based Imitation Learning},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  address = {Paris, France},
  month = may,
  year = {2020},
  doi = {10.1109/ICRA40945.2020.9197172},
}

Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision

Karttikeya Mangalam, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, and Juan Carlos Niebles

In IEEE Winter Conference on Applications of Computer Vision (WACV). Colorado, USA. Mar 2020

DOI arXiv Bib PDF Video Poster Website Talk

@inproceedings{Mangalam_WACV_2020,
  author = {Mangalam, Karttikeya and Adeli, Ehsan and Lee, Kuan-Hui and Gaidon, Adrien and Niebles, Juan Carlos},
  title = {Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  address = {Colorado, USA},
  month = mar,
  year = {2020},
  doi = {10.1109/WACV45572.2020.9093350},
}

Oral

Adversarial Cross-Domain Action Recognition with Co-Attention

Boxiao Pan, Zhangjie Cao, Ehsan Adeli, and Juan Carlos Niebles

In AAAI Conference on Artificial Intelligence (AAAI). New York. Feb 2020

Awarded DOI arXiv Bib PDF Poster Slides

Oral

@inproceedings{Pan_AAAI_2020,
  author = {Pan, Boxiao and Cao, Zhangjie and Adeli, Ehsan and Niebles, Juan Carlos},
  title = {Adversarial Cross-Domain Action Recognition with Co-Attention},
  booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
  address = {New York},
  month = feb,
  year = {2020},
  doi = {10.1609/aaai.v34i07.6854},
}

2019

Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning

De-An Huang, Danfei Xu, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Macau, China. Nov 2019

DOI arXiv Bib Video Slides

@inproceedings{Huang_IROS_2019,
  author = {Huang, De-An and Xu, Danfei and Zhu, Yuke and Garg, Animesh and Savarese, Silvio and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  address = {Macau, China},
  month = nov,
  year = {2019},
  doi = {10.1109/IROS40897.2019.8967761},
}

Learning Temporal Action Proposals with Fewer Labels

Jingwei Ji, Kaidi Cao, and Juan Carlos Niebles

In IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea. Oct 2019

DOI arXiv Bib PDF Poster

@inproceedings{Ji_ICCV_2019,
  author = {Ji, Jingwei and Cao, Kaidi and Niebles, Juan Carlos},
  title = {Learning Temporal Action Proposals with Fewer Labels},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Seoul, South Korea},
  month = oct,
  year = {2019},
  doi = {10.1109/ICCV.2019.00717},
}

Imitation Learning for Human Pose Prediction

Borui Wang, Ehsan Adeli, Hsu-kuang Chiu, De-An Huang, and Juan Carlos Niebles

In IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea. Oct 2019

DOI arXiv Bib PDF Poster

@inproceedings{Wang_ICCV_2019,
  author = {Wang, Borui and Adeli, Ehsan and Chiu, Hsu-kuang and Huang, De-An and Niebles, Juan Carlos},
  title = {Imitation Learning for Human Pose Prediction},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  address = {Seoul, South Korea},
  month = oct,
  year = {2019},
  doi = {10.1109/ICCV.2019.00722},
}

Oral

Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration

De-An Huang, Suraj Nair, Danfei Xu, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, California, USA. Jun 2019

Awarded DOI arXiv Bib PDF Video Poster Slides Website

Oral

@inproceedings{Huang_CVPR_2019,
  author = {Huang, De-An and Nair, Suraj and Xu, Danfei and Zhu, Yuke and Garg, Animesh and Fei-Fei, Li and Savarese, Silvio and Niebles, Juan Carlos},
  title = {Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Long Beach, California, USA},
  month = jun,
  year = {2019},
  doi = {10.1109/CVPR.2019.00876},
}

D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation

Chien-Yi Chang, De-An Huang, Yanan Sui, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, California, USA. Jun 2019

DOI arXiv Bib PDF Video Poster Slides Website

@inproceedings{Chang_CVPR_2019,
  author = {Chang, Chien-Yi and Huang, De-An and Sui, Yanan and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {{D3TW}: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Long Beach, California, USA},
  month = jun,
  year = {2019},
  doi = {10.1109/CVPR.2019.00366},
}

Peeking into the Future: Predicting Future Person Activities and Locations in Videos

Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander Hauptmann, and Li Fei-Fei

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, California, USA. Jun 2019

DOI arXiv Bib PDF Video Code Website

@inproceedings{Liang_CVPR_2019,
  author = {Liang, Junwei and Jiang, Lu and Niebles, Juan Carlos and Hauptmann, Alexander and Fei-Fei, Li},
  title = {Peeking into the Future: Predicting Future Person Activities and Locations in Videos},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Long Beach, California, USA},
  month = jun,
  year = {2019},
  doi = {10.1109/CVPR.2019.00587},
}

Action-Agnostic Human Pose Forecasting

Hsu-Kuang Chiu, Ehsan Adeli, Borui Wang, De-An Huang, and Juan Carlos Niebles

In IEEE Winter Conference on Applications of Computer Vision (WACV). Hawaii, USA. Jan 2019

DOI arXiv Bib Video Code Poster

@inproceedings{Chiu_WACV_2019,
  author = {Chiu, Hsu-Kuang and Adeli, Ehsan and Wang, Borui and Huang, De-An and Niebles, Juan Carlos},
  title = {Action-Agnostic Human Pose Forecasting},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  address = {Hawaii, USA},
  month = jan,
  year = {2019},
  doi = {10.1109/WACV.2019.00156},
}

Interpretable Visual Question Answering by Visual Grounding from Attention Supervision Mining

Yundong Zhang, Juan Carlos Niebles, and Alvaro Soto

In IEEE Winter Conference on Applications of Computer Vision (WACV). Hawaii, USA. Jan 2019

DOI arXiv Bib Poster Slides

@inproceedings{Zhang_WACV_2019,
  author = {Zhang, Yundong and Niebles, Juan Carlos and Soto, Alvaro},
  title = {Interpretable Visual Question Answering by Visual Grounding from Attention Supervision Mining},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  address = {Hawaii, USA},
  month = jan,
  year = {2019},
  doi = {10.1109/WACV.2019.00043},
}

2018

Learning to Decompose and Disentangle Representations for Video Prediction

Jun-Ting Hsieh, Bingbin Liu, De-An Huang, Li Fei-Fei, and Juan Carlos Niebles

In Conference on Neural Information Processing Systems (NeurIPS). Montreal, Canada. Dec 2018

arXiv Bib PDF Video Code Poster

@inproceedings{Hsieh_NeurIPS_2018,
  author = {Hsieh, Jun-Ting and Liu, Bingbin and Huang, De-An and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Learning to Decompose and Disentangle Representations for Video Prediction},
  booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
  address = {Montreal, Canada},
  month = dec,
  year = {2018},
}

Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation

Xiaoxue Zang, Ashwini Pokle, Marynel Vázquez, Kevin Chen, Juan Carlos Niebles, Alvaro Soto, and Silvio Savarese

In Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium. Nov 2018

DOI arXiv Bib PDF Code Website Data

@inproceedings{Zang_EMNLP_2018,
  author = {Zang, Xiaoxue and Pokle, Ashwini and Vázquez, Marynel and Chen, Kevin and Niebles, Juan Carlos and Soto, Alvaro and Savarese, Silvio},
  title = {Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation},
  booktitle = {Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  address = {Brussels, Belgium},
  month = nov,
  year = {2018},
  doi = {10.18653/v1/D18-1286},
}

Oral

End-to-End Joint Semantic Segmentation of Actors and Actions in Video

Jingwei Ji, Shyamal Buch, Alvaro Soto, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Munich, Germany. Sep 2018

Awarded DOI Bib PDF Video Code Poster Slides Talk

Oral

@inproceedings{Ji_ECCV_2018,
  author = {Ji, Jingwei and Buch, Shyamal and Soto, Alvaro and Niebles, Juan Carlos},
  title = {End-to-End Joint Semantic Segmentation of Actors and Actions in Video},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Munich, Germany},
  month = sep,
  year = {2018},
  doi = {10.1007/978-3-030-01225-0_43},
}

Temporal Modular Networks for Retrieving Complex Compositional Activities in Video

Bingbin Liu, Serena Yeung, Edward Chou, De-An Huang, Li Fei-Fei, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Munich, Germany. Sep 2018

DOI Bib PDF Video Poster Website

@inproceedings{Liu_ECCV_2018,
  author = {Liu, Bingbin and Yeung, Serena and Chou, Edward and Huang, De-An and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Temporal Modular Networks for Retrieving Complex Compositional Activities in Video},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Munich, Germany},
  month = sep,
  year = {2018},
  doi = {10.1007/978-3-030-01219-9_34},
}

Graph Distillation for Action Detection with Privileged Modalities

Zelun Luo, Lu Jiang, Jun-Ting Hsieh, Juan Carlos Niebles, and Li Fei-Fei

In European Conference on Computer Vision (ECCV). Munich, Germany. Sep 2018

DOI arXiv Bib PDF Poster Website

@inproceedings{Luo_ECCV_2018,
  author = {Luo, Zelun and Jiang, Lu and Hsieh, Jun-Ting and Niebles, Juan Carlos and Fei-Fei, Li},
  title = {Graph Distillation for Action Detection with Privileged Modalities},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Munich, Germany},
  month = sep,
  year = {2018},
  doi = {10.1007/978-3-030-01264-9_11},
}

Liquid Pouring Monitoring via Rich Sensory Inputs

Tz-Ying Wu, Juan-Ting Lin, Tsun-Hsuang Wang, Chan-Wei Hu, Juan Carlos Niebles, and Min Sun

In European Conference on Computer Vision (ECCV). Munich, Germany. Sep 2018

DOI arXiv Bib PDF Video Poster Website

@inproceedings{Wu_ECCV_2018,
  author = {Wu, Tz-Ying and Lin, Juan-Ting and Wang, Tsun-Hsuang and Hu, Chan-Wei and Niebles, Juan Carlos and Sun, Min},
  title = {Liquid Pouring Monitoring via Rich Sensory Inputs},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Munich, Germany},
  month = sep,
  year = {2018},
  doi = {10.1007/978-3-030-01252-6_21},
}

Oral

Finding ’It’: Weakly-Supervised Reference-Aware Visual Grounding in Instructional Video

De-An Huang, Shyamal Buch, Lucio Dery, Animesh Garg, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, Utah, USA. Jun 2018

Awarded DOI Bib PDF Poster Website Talk

Oral

@inproceedings{Huang_CVPR_2018b,
  author = {Huang, De-An and Buch, Shyamal and Dery, Lucio and Garg, Animesh and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Finding 'It': Weakly-Supervised Reference-Aware Visual Grounding in Instructional Video},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Salt Lake City, Utah, USA},
  month = jun,
  year = {2018},
  doi = {10.1109/CVPR.2018.00623},
}

Spotlight

What Makes a Video a Video: Analyzing Temporal Information in Video Understanding Models and Datasets

De-An Huang, Vignesh Ramanathan, Dhruv Mahajan, Lorenzo Torresani, Manohar Paluri, Li Fei-Fei, and Juan Carlos Niebles

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, Utah, USA. Jun 2018

Awarded DOI Bib PDF Poster Slides Talk

Spotlight

@inproceedings{Huang_CVPR_2018a,
  author = {Huang, De-An and Ramanathan, Vignesh and Mahajan, Dhruv and Torresani, Lorenzo and Paluri, Manohar and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {What Makes a Video a Video: Analyzing Temporal Information in Video Understanding Models and Datasets},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Salt Lake City, Utah, USA},
  month = jun,
  year = {2018},
  doi = {10.1109/CVPR.2018.00769},
}

A Deep Learning Based Behavioral Approach to Indoor Autonomous Navigation

Gabriel Sepúlveda, Juan Carlos Niebles, and Alvaro Soto

In IEEE International Conference on Robotics and Automation (ICRA). Brisbane, Australia. May 2018

DOI arXiv Bib Poster

@inproceedings{Sepulveda_ICRA_2018,
  author = {Sepúlveda, Gabriel and Niebles, Juan Carlos and Soto, Alvaro},
  title = {A Deep Learning Based Behavioral Approach to Indoor Autonomous Navigation},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  address = {Brisbane, Australia},
  month = may,
  year = {2018},
  doi = {10.1109/ICRA.2018.8460646},
}

Vision-Based Construction Activity Analysis in Long Video Sequences via Hidden Markov Models: Experiments on Earthmoving Operations

Dominic Roberts, Mani Golparvar-Fard, Juan Carlos Niebles, JunYoung Gwak, and Ruxiao Bao

In Construction Research Congress (CRC). New Orleans. Apr 2018

DOI Bib

@inproceedings{Roberts_CRC_2018,
  author = {Roberts, Dominic and Golparvar-Fard, Mani and Niebles, Juan Carlos and Gwak, JunYoung and Bao, Ruxiao},
  title = {Vision-Based Construction Activity Analysis in Long Video Sequences via Hidden Markov Models: Experiments on Earthmoving Operations},
  booktitle = {Construction Research Congress (CRC)},
  pages = {164-173},
  doi = {10.1061/9780784481288.017},
  month = apr,
  year = {2018},
  address = {New Orleans},
}

Behavioral Indoor Navigation With Natural Language Directions

Xiaoxue Zang, Marynel Vázquez, Juan Carlos Niebles, Alvaro Soto, and Silvio Savarese

In ACM/IEEE International Conference on Human Robot Interaction - Late Breaking Reports (HRI LBR). Chicago, Illinois, USA. Mar 2018

DOI Bib Poster

@inproceedings{Zang_HRILBR_2018,
  author = {Zang, Xiaoxue and Vázquez, Marynel and Niebles, Juan Carlos and Soto, Alvaro and Savarese, Silvio},
  title = {Behavioral Indoor Navigation With Natural Language Directions},
  booktitle = {ACM/IEEE International Conference on Human Robot Interaction - Late Breaking Reports (HRI LBR)},
  address = {Chicago, Illinois, USA},
  month = mar,
  year = {2018},
  doi = {10.1145/3173386.3177001},
}

2017

Spotlight

Visual Forecasting by Imitating Dynamics in Natural Sequences

Kuo-Hao Zeng, William B. Shen, De-An Huang, Min Sun, and Juan Carlos Niebles

In IEEE International Conference on Computer Vision (ICCV). Venice, Italy. Oct 2017

Awarded DOI arXiv Bib PDF Poster Talk

Spotlight

@inproceedings{Zeng_ICCV_2017,
  author = {Zeng, Kuo-Hao and Shen, William B. and Huang, De-An and Sun, Min and Niebles, Juan Carlos},
  title = {Visual Forecasting by Imitating Dynamics in Natural Sequences},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  address = {Venice, Italy},
  month = oct,
  year = {2017},
  doi = {10.1109/ICCV.2017.326},
}

Dense-Captioning Events in Videos

Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles

In IEEE International Conference on Computer Vision (ICCV). Venice, Italy. Oct 2017

DOI arXiv Bib PDF Video Code Website Data

@inproceedings{Krishna_ICCV_2017,
  author = {Krishna, Ranjay and Hata, Kenji and Ren, Frederic and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Dense-Captioning Events in Videos},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  address = {Venice, Italy},
  month = oct,
  year = {2017},
  doi = {10.1109/ICCV.2017.83},
}

Oral

End-to-End, Single-Stream Temporal Action Detection in Untrimmed Videos

Shyamal Buch, Victor Escorcia, Bernard Ghanem, Li Fei-Fei, and Juan Carlos Niebles

In British Machine Vision Conference (BMVC). London, UK. Sep 2017

Awarded Bib PDF Code Poster Talk

Oral

@inproceedings{Buch_BMVC_2017,
  author = {Buch, Shyamal and Escorcia, Victor and Ghanem, Bernard and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {End-to-End, Single-Stream Temporal Action Detection in Untrimmed Videos},
  booktitle = {British Machine Vision Conference (BMVC)},
  address = {London, UK},
  month = sep,
  year = {2017},
}

SST: Single-Stream Temporal Action Proposals

Shyamal Buch, Victor Escorcia, Chuanqi Shen, Bernard Ghanem, and Juan Carlos Niebles

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii, USA. Jun 2017

DOI Bib PDF Code Poster

@inproceedings{Buch_CVPR_2017,
  author = {Buch, Shyamal and Escorcia, Victor and Shen, Chuanqi and Ghanem, Bernard and Niebles, Juan Carlos},
  title = {{SST}: Single-Stream Temporal Action Proposals},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Honolulu, Hawaii, USA},
  month = jun,
  year = {2017},
  doi = {10.1109/CVPR.2017.675},
}

Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos

De-An Huang, Joseph Lim, Li Fei-Fei, and Juan Carlos Niebles

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii, USA. Jun 2017

DOI arXiv Bib PDF Poster Website

@inproceedings{Huang_CVPR_2017,
  author = {Huang, De-An and Lim, Joseph and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Honolulu, Hawaii, USA},
  month = jun,
  year = {2017},
  doi = {10.1109/CVPR.2017.116},
}

Spotlight

Agent-centric Risk Assessment: Accident Anticipation and Risky Region Localization

Kuo-Hao Zeng, Shih-Han Chou, Fu-Hsiang Chan, Juan Carlos Niebles, and Min Sun

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii, USA. Jun 2017

Awarded DOI arXiv Bib PDF Video Poster Website Data

Spotlight

@inproceedings{Zeng_CVPR_2017,
  author = {Zeng, Kuo-Hao and Chou, Shih-Han and Chan, Fu-Hsiang and Niebles, Juan Carlos and Sun, Min},
  title = {Agent-centric Risk Assessment: Accident Anticipation and Risky Region Localization},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Honolulu, Hawaii, USA},
  month = jun,
  year = {2017},
  doi = {10.1109/CVPR.2017.146},
}

Sparse composition of body poses and atomic actions for human activity recognition in RGB-D videos

Ivan Lillo, Juan Carlos Niebles, and Alvaro Soto

Image and Vision Computing. Mar 2017

DOI Bib Data

@article{Lillo_IVC_2017,
  author = {Lillo, Ivan and Niebles, Juan Carlos and Soto, Alvaro},
  title = {Sparse composition of body poses and atomic actions for human activity recognition in {RGB-D} videos},
  journal = {Image and Vision Computing},
  volume = {59},
  month = mar,
  year = {2017},
  doi = {10.1016/j.imavis.2016.11.004},
}

Leveraging Video Descriptions to Learn Video Question Answering

Kuo-Hao Zeng, Tseng-Hung Chen, Ching-Yao Chuang, Yuan-Hong Liao, Juan Carlos Niebles Sun, and Min

In AAAI Conference on Artificial Intelligence (AAAI). San Francisco, USA. Feb 2017

DOI arXiv Bib PDF Website Data

@inproceedings{Zeng_AAAI_2017,
  author = {Zeng, Kuo-Hao and Chen, Tseng-Hung and Chuang, Ching-Yao and Liao, Yuan-Hong and Sun, Juan Carlos Niebles and Min},
  title = {Leveraging Video Descriptions to Learn Video Question Answering},
  booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
  address = {San Francisco, USA},
  month = feb,
  year = {2017},
  doi = {10.1609/aaai.v31i1.11238},
}

2016

Connectionist Temporal Modeling for Weakly Supervised Action Labeling

De-An Huang, Li Fei-Fei, and Juan Carlos Niebles

In European Conference on Computer Vision (ECCV). Amsterdam, Netherlands. Oct 2016

DOI arXiv Bib Video Code Poster

@inproceedings{Huang_ECCV_2016,
  author = {Huang, De-An and Fei-Fei, Li and Niebles, Juan Carlos},
  title = {Connectionist Temporal Modeling for Weakly Supervised Action Labeling},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Amsterdam, Netherlands},
  month = oct,
  year = {2016},
  doi = {10.1007/978-3-319-46493-0_9},
}

DAPs: Deep Action Proposals for Action Understanding

Victor Escorcia, Fabian Caba Heilbron, Juan Carlos Niebles, and Bernard Ghanem

In European Conference on Computer Vision (ECCV). Amsterdam, Netherlands. Oct 2016

DOI Bib Video Code Poster Website

@inproceedings{Escorcia_ECCV_2016,
  author = {Escorcia, Victor and Caba Heilbron, Fabian and Niebles, Juan Carlos and Ghanem, Bernard},
  title = {{DAPs}: Deep Action Proposals for Action Understanding},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Amsterdam, Netherlands},
  month = oct,
  year = {2016},
  doi = {10.1007/978-3-319-46487-9_47},
}

Title Generation for User Generated Videos

Kuo-Hao Zeng, Tseng-Hung Chen, Juan Carlos Niebles, and Min Sun

In European Conference on Computer Vision (ECCV). Amsterdam, Netherlands. Oct 2016

DOI arXiv Bib Video Poster Website Data

@inproceedings{Zeng_ECCV_2016,
  author = {Zeng, Kuo-Hao and Chen, Tseng-Hung and Niebles, Juan Carlos and Sun, Min},
  title = {Title Generation for User Generated Videos},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Amsterdam, Netherlands},
  month = oct,
  year = {2016},
  doi = {10.1007/978-3-319-46475-6_38},
}

Fast Temporal Activity Proposals for Efficient Detection of Human Actions in Untrimmed Videos

Fabian Caba Heilbron, Juan Carlos Niebles, and Bernard Ghanem

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA. Jun 2016

DOI Bib PDF Code Website

@inproceedings{Heilbron_CVPR_2016,
  author = {Caba Heilbron, Fabian and Niebles, Juan Carlos and Ghanem, Bernard},
  title = {Fast Temporal Activity Proposals for Efficient Detection of Human Actions in Untrimmed Videos},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Las Vegas, USA},
  month = jun,
  year = {2016},
  doi = {10.1109/CVPR.2016.211},
}

A Hierarchical Pose-Based Approach to Complex Action Understanding Using Dictionaries of Actionlets and Motion Poselets

Ivan Lillo, Juan Carlos Niebles, and Alvaro Soto

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA. Jun 2016

DOI arXiv Bib PDF Data

@inproceedings{Lillo_CVPR_2016,
  author = {Lillo, Ivan and Niebles, Juan Carlos and Soto, Alvaro},
  title = {A Hierarchical Pose-Based Approach to Complex Action Understanding Using Dictionaries of Actionlets and Motion Poselets},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Las Vegas, USA},
  month = jun,
  year = {2016},
  doi = {10.1109/CVPR.2016.218},
}

2015

ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding

Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA. Jun 2015

DOI Bib PDF Code Website Data

@inproceedings{Caba_CVPR_2015,
  author = {Caba Heilbron, Fabian and Escorcia, Victor and Ghanem, Bernard and Niebles, Juan Carlos},
  title = {{ActivityNet}: A Large-Scale Video Benchmark for Human Activity Understanding},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Boston, USA},
  month = jun,
  year = {2015},
  doi = {10.1109/CVPR.2015.7298698},
}

On the Relationship between Visual Attributes and Convolutional Networks

Victor Escorcia, Juan Carlos Niebles, and Bernard Ghanem

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA. Jun 2015

DOI Bib PDF Video

@inproceedings{Escorcia_CVPR_2015,
  author = {Escorcia, Victor and Niebles, Juan Carlos and Ghanem, Bernard},
  title = {On the Relationship between Visual Attributes and Convolutional Networks},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Boston, USA},
  month = jun,
  year = {2015},
  doi = {10.1109/CVPR.2015.7298730},
}

Robust Manhattan Frame Estimation from a Single RGB-D Image

Bernard Ghanem, Ali Thabet, Juan Carlos Niebles, and Fabian Caba Heilbron

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA. Jun 2015

DOI Bib PDF

@inproceedings{Ghanem_CVPR_2015,
  author = {Ghanem, Bernard and Thabet, Ali and Niebles, Juan Carlos and Caba Heilbron, Fabian},
  title = {Robust Manhattan Frame Estimation from a Single {RGB-D} Image},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Boston, USA},
  month = jun,
  year = {2015},
  doi = {10.1109/CVPR.2015.7299001},
}

2014

Camera Motion and Surrounding Scene Appearance as Context for Action Recognition

Fabian Caba Heilbron, Ali Thabet, Juan Carlos Niebles, and Bernard Ghanem

In Asian Conference on Computer Vision (ACCV). Singapore. Nov 2014

DOI Bib Website

@inproceedings{Heilbron_ACCV_2014,
  author = {Heilbron, Fabian Caba and Thabet, Ali and Niebles, Juan Carlos and Ghanem, Bernard},
  title = {Camera Motion and Surrounding Scene Appearance as Context for Action Recognition},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  address = {Singapore},
  month = nov,
  year = {2014},
  doi = {10.1007/978-3-319-16817-3_38},
}

Vision-based workface assessment using depth images for activity analysis of interior construction operations

Ardalan Khosrowpour, Juan Carlos Niebles, and Mani Golparvar-Fard

Automation in Construction. Dec 2014

DOI Bib

@article{Khosrowpour_AutCon_2014,
  author = {Khosrowpour, Ardalan and Niebles, Juan Carlos and Golparvar-Fard, Mani},
  title = {Vision-based workface assessment using depth images for activity analysis of interior construction operations},
  journal = {Automation in Construction},
  volume = {48},
  month = dec,
  year = {2014},
  doi = {10.1016/j.autcon.2014.08.003},
}

Discriminative Hierarchical Modeling of Spatio-Temporally Composable Human Activities

Ivan Lillo, Alvaro Soto, and Juan Carlos Niebles

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Colombus, USA. Jun 2014

DOI Bib Video Website Data

@inproceedings{Lillo_CVPR_2014,
  author = {Lillo, Ivan and Soto, Alvaro and Niebles, Juan Carlos},
  title = {Discriminative Hierarchical Modeling of Spatio-Temporally Composable Human Activities},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Colombus, USA},
  month = jun,
  year = {2014},
  doi = {10.1109/CVPR.2014.109},
}

Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from long sequences of RGB-D Images

Ardalan Khosrowpour, Igor Fedorov, Aleksander Holynski, Juan Carlos Niebles, and Mani Golparvar-Fard

In Construction Research Congress (CRC). Atlanta, USA. May 2014

DOI Bib

@inproceedings{Khosrowpour_CRC_2014,
  author = {Khosrowpour, Ardalan and Fedorov, Igor and Holynski, Aleksander and Niebles, Juan Carlos and Golparvar-Fard, Mani},
  title = {Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from long sequences of {RGB-D} Images},
  booktitle = {Construction Research Congress (CRC)},
  month = may,
  year = {2014},
  address = {Atlanta, USA},
  doi = {10.1061/9780784413517.075},
}

Collecting and Annotating Human Activities in Web Videos

Fabian Caba Heilbron, and Juan Carlos Niebles

In ACM International Conference on Multimedia Retrieval (ICMR). Glasgow, UK. Apr 2014

DOI Bib Code

@inproceedings{Heilbron_ICMR_2014,
  author = {Caba Heilbron, Fabian and Niebles, Juan Carlos},
  title = {Collecting and Annotating Human Activities in Web Videos},
  booktitle = {ACM International Conference on Multimedia Retrieval (ICMR)},
  address = {Glasgow, UK},
  month = apr,
  year = {2014},
  doi = {10.1145/2578726.2578775},
}

2013

Spatio-Temporal Human-Object Interactions for Action Recognition in Videos

Victor Escorcia, and Juan Carlos Niebles

In IEEE International Conference on Computer Vision Workshops (ICCVW). Sydney, Australia. Dec 2013

DOI Bib PDF Video Website

@inproceedings{Escorcia_ICCVW_2013,
  author = {Escorcia, Victor and Niebles, Juan Carlos},
  title = {Spatio-Temporal Human-Object Interactions for Action Recognition in Videos},
  booktitle = {IEEE International Conference on Computer Vision Workshops (ICCVW)},
  address = {Sydney, Australia},
  month = dec,
  year = {2013},
  doi = {10.1109/ICCVW.2013.72},
}

Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers

Mani Golparvar-Fard, Arsalan Heydarian, and Juan Carlos Niebles

Advanced Engineering Informatics. Oct 2013

DOI Bib

@article{GolparvarFard_AEI_2013,
  author = {Golparvar-Fard, Mani and Heydarian, Arsalan and Niebles, Juan Carlos},
  title = {Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers},
  journal = {Advanced Engineering Informatics},
  volume = {27},
  number = {4},
  month = oct,
  year = {2013},
  doi = {10.1016/j.aei.2013.09.001},
}

Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors

Milad Memarzadeh, Mani Golparvar-Fard, and Juan Carlos Niebles

Automation in Construction. Jul 2013

DOI Bib

@article{Memarzadeh_AutCon_2013,
  author = {Memarzadeh, Milad and Golparvar-Fard, Mani and Niebles, Juan Carlos},
  title = {Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors},
  journal = {Automation in Construction},
  volume = {32},
  month = jul,
  year = {2013},
  doi = {10.1016/j.autcon.2012.12.002},
}

2012

Automated Vision-based Recognition of Construction Worker Actions for Building Interior Construction Operations Using RGBD Cameras

Victor Escorcia, Maria A. Dávila, Mani Golparvar-Fard, and Juan Carlos Niebles

In Construction Research Congress (CRC). West Lafayette, Indiana, USA. May 2012

DOI Bib

@inproceedings{Escorcia_CRC_2012,
  author = {Escorcia, Victor and Dávila, Maria A. and Golparvar-Fard, Mani and Niebles, Juan Carlos},
  title = {Automated Vision-based Recognition of Construction Worker Actions for Building Interior Construction Operations Using {RGBD} Cameras},
  booktitle = {Construction Research Congress (CRC)},
  address = {West Lafayette, Indiana, USA},
  month = may,
  year = {2012},
  doi = {10.1061/9780784412329.089},
}

Automated visual recognition of construction equipment actions using spatio-temporal features and multiple binary support vector machines

Arsalan Heydarian, Mani Golparvar-Fard, and Juan Carlos Niebles

In Construction Research Congress (CRC). West Lafayette, Indiana, USA. May 2012

DOI Bib

@inproceedings{Heydarian_CRC_2012,
  author = {Heydarian, Arsalan and Golparvar-Fard, Mani and Niebles, Juan Carlos},
  title = {Automated visual recognition of construction equipment actions using spatio-temporal features and multiple binary support vector machines},
  booktitle = {Construction Research Congress (CRC)},
  address = {West Lafayette, Indiana, USA},
  month = may,
  year = {2012},
  doi = {10.1061/9780784412329.090},
}

Real-time and automated recognition and 2D tracking of Construction workers and equipment from Site video streams

Milad Memarzadeh, Arsalan Heydarian, Mani Golparvar-Fard, and Juan Carlos Niebles

In ASCE International Conference on Computing in Civil Engineering (i3CE). Clearwater Beach, Florida, USA. Jun 2012

DOI Bib

@inproceedings{Memarzadeh_i3CE_2012,
  author = {Memarzadeh, Milad and Heydarian, Arsalan and Golparvar-Fard, Mani and Niebles, Juan Carlos},
  title = {Real-time and automated recognition and {2D} tracking of Construction workers and equipment from Site video streams},
  booktitle = {ASCE International Conference on Computing in Civil Engineering (i3CE)},
  address = {Clearwater Beach, Florida, USA},
  month = jun,
  year = {2012},
  doi = {10.1061/9780784412343.0054},
}

2010

Oral

Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification

Juan Carlos Niebles, Chih-Wei Chen, and Li Fei-Fei

In European Conference on Computer Vision (ECCV). Hersonissos, Crete, Greece. Sep 2010

Awarded DOI Bib PDF Slides Data

Oral

@inproceedings{Niebles_ECCV_2010,
  author = {Niebles, Juan Carlos and Chen, Chih-Wei and Fei-Fei, Li},
  title = {Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Hersonissos, Crete, Greece},
  month = sep,
  year = {2010},
  doi = {10.1007/978-3-642-15552-9_29},
}

Efficient Extraction of Human Motion Volumes by Tracking

Juan Carlos Niebles, Bohyung Han, and Li Fei-Fei

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). San Francisco, USA. Jun 2010

DOI Bib PDF

@inproceedings{Niebles_CVPR_2010,
  author = {Niebles, Juan Carlos and Han, Bohyung and Fei-Fei, Li},
  title = {Efficient Extraction of Human Motion Volumes by Tracking},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {San Francisco, USA},
  month = jun,
  year = {2010},
  doi = {10.1109/CVPR.2010.5540152},
}

2009

Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition

Bangpeng Yao, Juan Carlos Niebles, and Li Fei-Fei

In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Miami, USA. Jun 2009

DOI Bib

@inproceedings{Yao_CVPRW_2009,
  author = {Yao, Bangpeng and Niebles, Juan Carlos and Fei-Fei, Li},
  title = {Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  address = {Miami, USA},
  month = jun,
  year = {2009},
  doi = {10.1109/CVPRW.2009.5204222}
}

2008

Extracting Moving People from Internet Videos

Juan Carlos Niebles, Bohyung Han, Andras Ferencz, and Li Fei-Fei

In European Conference on Computer Vision (ECCV). Marseilles, France. Oct 2008

DOI Bib PDF Poster Website Data

@inproceedings{Niebles_ECCV_2008,
  author = {Niebles, Juan Carlos and Han, Bohyung and Ferencz, Andras and Fei-Fei, Li},
  title = {Extracting Moving People from Internet Videos},
  booktitle = {European Conference on Computer Vision (ECCV)},
  address = {Marseilles, France},
  month = oct,
  year = {2008},
  doi = {10.1007/978-3-540-88693-8_39},
}

Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words

Juan Carlos Niebles, Hongcheng Wang, and Li Fei-Fei

International Journal of Computer Vision (IJCV). Sep 2008

DOI Bib Website

@article{Niebles_IJCV_2008,
  author = {Niebles, Juan Carlos and Wang, Hongcheng and Fei-Fei, Li},
  title = {Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words},
  journal = {International Journal of Computer Vision (IJCV)},
  volume = {79},
  month = sep,
  year = {2008},
  doi = {10.1007/s11263-007-0122-4},
}

Spatial-Temporal Correlatons for Unsupervised Action Classification

Silvio Savarese, Andrey Del Pozo, Juan Carlos Niebles, and Li Fei-Fei

In IEEE Workshop on Motion and Video Computing (WMVC). Copper Mountain, Colorado, USA. Jan 2008

DOI Bib

@inproceedings{Savarese_WMVC_2008,
  author = {Savarese, Silvio and Pozo, Andrey Del and Niebles, Juan Carlos and Fei-Fei, Li},
  title = {Spatial-Temporal Correlatons for Unsupervised Action Classification},
  booktitle = {IEEE Workshop on Motion and Video Computing (WMVC)},
  address = {Copper Mountain, Colorado, USA},
  month = jan,
  year = {2008},
  doi = {10.1109/WMVC.2008.4544068},
}

2007

OPTIMOL: a framework for Online Picture collecTion via Incremental MOdel Learning

Li-Jia Li, Juan Carlos Niebles, and Li Fei-Fei

In Association for the Advancement of Artificial Intelligence (AAAI) 2007 Robot Competition and Exhibition. Vancouver, Canada. Jul 2007

Bib PDF Website

@inproceedings{Li_AAAI_2007,
  author = {Li, Li-Jia and Niebles, Juan Carlos and Fei-Fei, Li},
  title = {{OPTIMOL}: a framework for Online Picture collecTion via Incremental MOdel Learning},
  booktitle = {Association for the Advancement of Artificial Intelligence (AAAI) 2007 Robot Competition and Exhibition},
  address = {Vancouver, Canada},
  month = jul,
  year = {2007},
}

A Hierarchical Model of Shape and Appearance for Human Action Classification

Juan Carlos Niebles, and Li Fei-Fei

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Minneapolis, USA. Jun 2007

DOI Bib

@inproceedings{Niebles_CVPR_2007,
  author = {Niebles, Juan Carlos and Fei-Fei, Li},
  title = {A Hierarchical Model of Shape and Appearance for Human Action Classification},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  address = {Minneapolis, USA},
  month = jun,
  year = {2007},
  doi = {10.1109/CVPR.2007.383132},
}

2006

Oral

Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words

Juan Carlos Niebles, Hongcheng Wang, and Li Fei-Fei

In British Machine Vision Conference (BMVC). Edinburgh, UK. Sep 2006

Awarded DOI Bib PDF Video Poster

Oral

@inproceedings{Niebles_BMVC_2006,
  author = {Niebles, Juan Carlos and Wang, Hongcheng and Fei-Fei, Li},
  title = {Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words},
  booktitle = {British Machine Vision Conference (BMVC)},
  address = {Edinburgh, UK},
  month = sep,
  year = {2006},
  doi = {10.5244/C.20.127},
}

2004

Signal Processing Unit for River Tugboat Telemetry System

Humberto Campanella, Mauricio Pardo, Victor Manotas, Javier Páez, Juan Carlos Niebles, and David Angulo

In Conference on Design of Circuits and Integrated Systems (DCIS). Bordeaux, France. Nov 2004

Bib

@inproceedings{Campanella_DCIS_2004,
  author = {Campanella, Humberto and Pardo, Mauricio and Manotas, Victor and P\'{a}ez, Javier and Niebles, Juan Carlos and Angulo, David},
  title = {Signal Processing Unit for River Tugboat Telemetry System},
  booktitle = {Conference on Design of Circuits and Integrated Systems (DCIS)},
  address = {Bordeaux, France},
  month = nov,
  year = {2004},
}

2001

Dispositivos automatas para navegacion, deteccion y recoleccion de pelotas de tenis en ambientes conocidos

Carlos Cubas, Marcos Consuegra, Royman Lopez, Alexis Sierra, Jorge Leon, Fernando Mendez, Carlos Ochoa, Fabian Panesso, Edgar Sotter, Fabian Manotas, and 4 more authors

Revista Ingenieria y Desarrollo (IyD). Jan 2001

Bib PDF

@article{Cubas_IyD_2001,
  author = {Cubas, Carlos and Consuegra, Marcos and Lopez, Royman and Sierra, Alexis and Leon, Jorge and Mendez, Fernando and Ochoa, Carlos and Panesso, Fabian and Sotter, Edgar and Manotas, Fabian and Manotas, Victor and Niebles, Juan Carlos and Pardo, Mauricio and Vallejo, Eric},
  title = {Dispositivos automatas para navegacion, deteccion y recoleccion de pelotas de tenis en ambientes conocidos},
  journal = {Revista Ingenieria y Desarrollo (IyD)},
  month = jan,
  year = {2001},
}