Title: History of Multimedia Retrieval & Research Directions in Real Time Multimodal Emergency Response Applications
Speaker: Alex Jaimes
Abstract: Image retrieval has been around for roughly 30 years, and the technical community, and industry, have come a long way: from retrieving simple syntax in the early 90s (textures, shapes, etc.), to web scale retrieval today (objects, scenes, locations, etc.). Yet, and in spite of the commoditization of many Computer Vision algorithms, we’re still far from achieving what the community set out to achieve long ago, and what we need for many real world applications. In this talk, first, I will give my personal perspective on the history of the field, where it is today, and where it needs to go from a technical point of view. Then, I will use emergency response as an example application that requires multi-modal, real-time retrieval combining many of the most exciting fields in AI: NLP, IR, CV, anomaly detection, and many others. When an emergency event, or an incident relevant for peacekeeping or humanitarian needs first occurs, getting the right information as quickly as possible is critical in saving lives. When an event is ongoing, information on what is happening can be critical in making decisions to keep people safe and take control of the particular situation unfolding. In both cases, first responders, peacekeepers, and others have to quickly make decisions that include what resources to deploy and where. Fortunately, in most emergencies, people share information on public platforms, while sensor data is increasingly becoming available. A platform to detect emergency situations and deliver the right information has to deal with ingesting thousands of noisy data points per second: sifting through and identifying relevant information, from different sources, in different formats, with varying levels of detail, in real time, so that relevant individuals and teams can be alerted at the right level and at the right time. I will describe the technical challenges in processing vast amounts of heterogeneous, noisy data in real time from the web and other sources, highlighting the importance of interdisciplinary research and a human-centered approach to address problems in humanitarian and emergency response.
Bio: Alex is Chief Scientist & SVP of AI at Dataminr. Alex is a leader in AI and as an Engineering executive and scientist has built and led AI teams at large companies such as Yahoo and at several startups, where he has led efforts to build AI products used by millions of people across multiple B2C and B2B industries (real-time event detection/emergency response, healthcare, self-driving cars, media, telecomm, etc.). He has 15+ years of intl. experience in research (Columbia U., KAIST) and product impact at scale (Yahoo, Telefónica, IBM, Fuji Xerox, Siemens, AT&T Bell Labs, DigitalOcean, and IDIAP-EPFL) in the USA, Japan, Chile, Switzerland, Spain, and South Korea. He has been a professor (KAIST, South Korea), and has 100+ patents and publications (h-index 40) in top tier conferences and journals in diverse topics in AI. His work has received 7K+ citations and he has been featured widely in the press (MIT Tech review, CNBC, Vice, TechCrunch, Yahoo! Finance, etc.). He has given 100+ invited talks at the top academic and industry conferences (UN AI for Good Global Summit, ICML & NeurIPs workshops, KDD, O’Reilly AI, Strata, Velocity, the Deep Learning Summit (Re-Work), Tech Open Air, the Future of Technology Summit, CogX, Stanford, Cornell, & Columbia Universities, etc.). He is a mentor at Endeavor (which leads the high-impact entrepreneurship movement around the world) and Techstars; he is a member of the advisory board of Digital Divide Data (a non-for profit that creates sustainable tech opportunities for underserved youth, their families, and their communities in Asia and Africa), and was an early voice in Human-Centered AI (Computing). He is one of ten experts in the Colombian Government’s Artificial Intelligence Expert Mission, which will evaluate and produce concrete recommendations in the short, medium and long term to implement an AI Policy. Colombia’s AI Expert Mission is one of the first of its kind in the region, and one of the first to focus on developing measures for the development of education and employment policies for the fourth industrial revolution. Alex is an active member of the research community (publishing and being in the program committee of several top-tier conferences). He holds a Ph.D. and an M.S. from Columbia University.
Title: Challenges in Multimodal Conversational Search and Recommendation
Speaker: Tat-Seng Chua
Abstract: Information search has been evolving from mostly unidirectional and text-based to interactive and multimodal. Recently, there is also a growing interest in all matters conversational. Multimodal conversation offers users a natural way to query the system by combining text/speech, images/videos and possibly gesture. It also helps to tackle the basic asymmetric problems by injecting conversation to help resolve the ambiguities in search and recommendation. However, the evolution from traditional IR to multimodal conversational search and recommendation (MCSR) faces many challenges. The first set of challenges touches on the basic MCSR models, including how to integrate task-oriented and open domain models, how to model multimodal context and history, and how to integrate domain knowledge and user models. The second set of challenges involves basic interactivity issues, including how to naturally converse using text and visual modalities, how to incorporate intervention strategy into search and browsing; and how to perform interactive IR, QA and recommendation. The third set of challenges looks more into the future on how to build dialogue simulator, and how to make MCSR systems extendable and active by allowing the system and users to co-evolve and becoming more intelligent together. This talk presents current research with pointers towards future research.
Bio: Dr. Chua is the KITHCT Chair Professor at the School of Computing, National University of Singapore (NUS). He is also the Distinguished Visiting Professor of Tsinghua University, Visiting Pao Yue-Kong Chair Professor of Zhejiang University, and Distinguished Visitng Professor of Sichuan University. Dr. Chua was the Founding Dean of the School of Computing from 1998-2000. His main research interests include unstructured data analytics, video analytics, conversational search and recommendation, and robust and trustable AI. Dr. Chua is the co- Director of NExT (a joint research Center between NUS and Tsinghua) and Sea-NExT (a joint Lab between Sea Group and NExT). Dr Chua is the recipient of the 2015 ACM SIGMM Achievements Award for the Outstanding Technical Contributions to Multimedia Computing, Communications and Applications. He is the Chair of steering committee of Multimedia Modeling (MMM) conference series, and ACM International Conference on Multimedia Retrieval (ICMR) (2015-2018). He was the General Co-Chair of ACM Multimedia 2005, ACM CIVR (now ACM ICMR) 2005, ACM SIGIR 2008, ACM Web Science 2015, ACM MM-Asia 2020, and the upcoming ACM conferences on WSDM 2023 and TheWebConf 2024. He serves in the editorial boards of three international journals. Dr. Chua is the co-Founder of two technology startup companies in Singapore. He holds a PhD from the University of Leeds, UK.