ASU engineering faculty win award with innovative 'person-centered' multimedia paradigm

December 13, 2017

When you update the software on your phone, it takes some effort to adapt to changes to the software you have become so used to. You have to learn how it works, but why shouldn’t the technology adapt to you as well?

A team of researchers in Arizona State University’s Ira A. Fulton Schools of Engineering set out to make this a reality by creating adaptive assistive computing technology. Troy McDaniel holding a device. Caption: Troy McDaniel shows off the haptic devices that will be placed on the chair to deliver vibration patterns. Photographer: Jessica Hochreiter/ASU Troy McDaniel shows off the haptic devices that will be placed on the chair to deliver vibration patterns. Photo by Jessica Hochreiter/ASU Download Full Image

This work caught the attention of the Institute of Electrical and Electronics Engineers, which presented three ASU faculty members and one ASU postdoctoral researcher the 2017 IEEE MultiMedia Best Department Article Award on Oct. 26, at the 25th ACM Multimedia conference in Mountain View, California at the Computer History Museum.

Executive Vice President of ASU Knowledge Enterprise Development Sethuraman Panchanathan, Assistant Professor Shayok Chakraborty of Florida State University (formerly an assistant research professor at ASU), Assistant Research Professor Troy McDaniel and Postdoctoral Research Associate Ramin Tadayon were the minds behind the award-winning article titled, “Person-centered multimedia computing: a new paradigm inspired by assistive and rehabilitative applications.”

The paper introduced two applications within a new paradigm developed by Panchanathan, the director of the Center for Cognitive Ubiquitous Computing, also known as CUbiC, called “person-centered multimedia computing.”

“It’s a paradigm that proposes ideal technological designs as individualized designs: technology that adapts to its users over time through continual use,” McDaniel said.

McDaniel, alongside Panchanathan, Chakraborty and Tadayon, focused on applying the new model to social assistive aids for individuals who are blind, and rehabilitative technologies for individuals who have suffered strokes.

“Designing for individuals with disabilities necessitates individualized designs because no two disabilities are the same,” McDaniel said. “The proposed paradigm stems from this need for person-centered designs.”

CUbiC’s core research in multimedia computing, machine learning, computer vision and haptics is driven by application areas of assistive, rehabilitative and healthcare technologies for individuals with disabilities.

“The challenge with individualized designs is that we could potentially spend a lot of time and energy building and fine-tuning a technology for a specific user, but what if someone else wants to use it?” McDaniel asked. “How do we find the balance to ensure solutions can generalize to other users?”

The researchers addressed this concern by exploring coadaptive algorithms, which allow both the person and the machine to adapt to each other over time.

“That’s what person-centeredness is all about — the technology adapting to you so that it's perfectly suited for you,” McDaniel said.

One application of the new paradigm is the Social Interaction Assistant, an assistive device that allows individuals who are blind access to nonverbal social cues, such as facial expressions. A body-worn or tabletop webcam would use machine learning and computer vision algorithms to recognize nonverbal cues of interaction partners as well as haptic, touch-based delivery devices for discreet, personal communication to an individual who is blind.

“The presentation part is difficult: you don’t want to obstruct a user’s hearing because then they can’t hear their partner speak,” McDaniel said. “So, we try to convey as much information as possible through the sense of touch. We can do that using, for example, vibrations. By exploring vibration patterns that vary both spatially and temporally, and haptic devices capable of delivering these cues, we have found a set of patterns that intuitively convey something as complex as facial expressions and emotions.”

Coadaptive algorithms can also be used to improve accuracy of machine learning and computer vision algorithms, like the ones used in this technology, by considering context and the behavior of the user.

A year after the article was published in August 2016, it won the Best Department Article award, assuring the researchers that they were on the right track.

“My impression is that people are recognizing the work and identifying something that’s an important and useful contribution,” McDaniel said.

McDaniel’s team is now applying person-centered multimedia computing to other applications.

“As we investigate more projects here in CUbiC, we’re developing them under this paradigm because it’s such a powerful approach,” McDaniel said. “It has inspired not only our current work but much of our planned future work as well, so we’re continuing to explore different applications that we can apply this paradigm to in a useful way.”

Student Science/Technology Writer, Ira A. Fulton Schools of Engineering

National Academy of Inventors announces pair of ASU researchers as 2017 fellows

December 13, 2017

Stephen Albert Johnston and Deirdre R. Meldrum of the Biodesign Institute at Arizona State University have been named fellows of the National Academy of Inventors (NAI), the organization recently announced.

Election to NAI Fellow status is the highest professional accolade bestowed solely to academic inventors who have demonstrated a prolific spirit of innovation in creating or facilitating outstanding inventions that have made a tangible impact on the quality of life, economic development, and welfare of society.  Stephen Albert Johnston and Deirdre R. Meldrum of the Biodesign Institute at Arizona State University have been named fellows of the National Academy of Inventors, the organization announced. Download Full Image

“The election of Drs. Johnston and Meldrum as NAI Fellows is a testament to the quality of far-reaching impact of the groundbreaking research taking place at ASU,” said Sethuraman “Panch” Panchanathan, executive vice president of Knowledge Enterprise Development and chief research and innovation officer at Arizona State University. “We hope that our work will continue to serve communities and change lives around the world.”

Johnston is director of the Biodesign Center for Innovations in Medicine, a professor in the School of Life Sciences and CEO of Calviri, Inc., an ASU startup.  

Johnston has experience in basic science, notably first cloning the Gal4 gene, showing that proteins have separable functional domains and discovering the AAA proteins and their role in transcription.

His focus now is in translational sciences and technology development.

Stephen Albert Johnston

“I think inventors are the most important drivers of the good things for humanity,” Johnston said. “So, I am honored that the NAI would include me in their ranks.”

In 2012, for example, Johnston, along with fellow ASU professor and co-founder Neal Woodbury, spun out a startup, HealthTell, built around a new liquid-biopsy technology.

In 2017, the Chinese company iCarbonX formed an alliance with HealthTell to make its ImmunoSignature technology available in China, Singapore and Taiwan. The innovation is part of $400 million international Digital Life Alliance aimed at producing a personalized health guide for consumers. 

From just a single drop of blood, this diagnostic powerhouse can detect diseases that involve an immune response (autoimmune, cancer, infectious disease, metabolic and neurologic diseases). 

In his career, Johnston was also co-inventor/innovator of pathogen derived resistance, endosperm balance number, organelle transformation, the gene gun, genetic immunization, TEV protease system, expression library immunization, linear expression elements, synbody therapeutics, immunosignature diagnostics and neoantigen preventative cancer vaccines. 

Johnston is author of over 170 journal articles, has over 25 patents and has garnered approximately $95M in grant support including large programs from DARPA, NIAID and NHLBI. He was named Arizona Bioscience Researcher of the Year in 2016.

Deirdre R. Meldrum

Meldrum is director of the Biodesign Center for Biosignatures Discovery Automation, distinguished professor of biosignatures discovery, professor of electrical engineering, and previous dean of the Ira A. Fulton Schools of Engineering at Arizona State University. Meldrum was also director and principal investigator of the NIH Center of Excellence in Genomic Sciences (CEGS): Microscale Life Sciences Center (MLSC). 

She is known for her ability to integrate multiple disciplines, including bioengineering, biomaterials, nanotechnology, electrical, chemical, and computer science engineering with fields such as biology, genomics, proteomics and single cell analyses to create new paths of discovery.

“An important result of invention is the societal impact achieved through commercialization,” Meldrum said. “Our inventions include new technologies and sensors for live single cell analysis that we plan to commercialize to positively impact human health as well as discoveries in the oceans. It is a real honor to join other creative innovators as a fellow of the NAI.”

She was a member for two terms of the National Advisory Council for Human Genome Research, U. S. Department of Health and Human Services and is currently a member of the Scientific Advisory Committee for the Department of Energy (DOE) Joint Genome Institute (JGI) and vice-chair of the Northern Arizona Healthcare Foundation. She is a fellow of the American Association for the Advancement of Science, the Institute of Electrical and Electronics Engineers and the American Institute for Medical and Biological Engineering.

Meldrum is author of 231 peer-reviewed publications including 122 journal articles, has 35 patents including 7 issued and has been awarded over $80M in external grant funding from the NIH, NSF and the Keck Foundation.

Johnston and Meldrum join several other ASU faculty, including Charles Arntzen, Bruce Rittmann, Stuart Lindsay, Michael Kozicki, and Sethuraman “Panch” Panchanathan who have been equally honored as NAI Fellows in recent years, helping to further raise ASU’s reputation as a leader in innovation. In addition, in another strategic effort to foster ASU innovation throughout its entrepreneurial ecosystem, ASU recently formed a NAI Chapter earlier this year. Launched in March 2017, it recognizes inventors and promotes innovation across all disciplines of the institution. 

Those elected to the rank of NAI Fellow were nominated by their peers for outstanding contributions to innovation in areas such as patents and licensing, innovative discovery and technology, significant impact on society, and support and enhancement of innovation.

On April 5, 2018, the 2017 NAI Fellows will be inducted as part of the Seventh Annual NAI Conference of the National Academy of Inventors at the Mayflower Hotel, Autograph Collection in Washington, D.C. Andrew H. Hirshfeld, U.S. Commissioner for Patents, will provide the keynote address for the induction ceremony.

The 2017 NAI Fellows will also be highlighted with a full-page announcement in The Chronicle of Higher Education Jan. 19, 2018 issue, and in an upcoming issue of Science and Technology and Innovation, journal of the National Academy of Inventors.

The 2017 class of NAI Fellows was evaluated by the 2017 Selection Committee, which included 18 members comprising NAI Fellows, U.S. National Medals recipients, National Inventors Hall of Fame inductees, members of the National Academies of Sciences, Engineering, and Medicine and senior officials from the USPTO, National Institute of Standards and Technology, Association of American Universities, American Association for the Advancement of Science, Association of Public and Land-grant Universities, Association of University Technology Managers, and National Inventors Hall of Fame, among other organizations. 

With the election of the 2017 class there are now 912 NAI Fellows, representing over 250 research universities and governmental and non-profit research institutes. The 2017 fellows are named inventors on nearly 6,000 issued U.S. patents, bringing the collective patents held by all NAI Fellows to more than 32,000 issued U.S. patents.

Included among all NAI Fellows are more than 100 presidents and senior leaders of research universities and non-profit research institutes; 439 members of the National Academies of Sciences, Engineering, and Medicine; 36 inductees of the National Inventors Hall of Fame; 52 recipients of the U.S. National Medal of Technology and Innovation and U.S. National Medal of Science; 29 Nobel Laureates; 261 AAAS Fellows; 168 IEEE Fellows; and 142 Fellows of the American Academy of Arts and Sciences, among other awards and distinctions.