World's First All-Optical Generative AI Chip Developed by Shanghai Jiao Tong University

On December 19, a research team led by Assistant Professor Chen Yitong from the Institute of Image Communication and Network Engineering, School of Integrated Circuits (School of Information and Electronic Engineering) at Shanghai Jiao Tong University achieved a major breakthrough in the field of next-generation computing chips. They have developed the first all-optical computing chip capable of supporting large-scale semantic media generative models. The related research, titled "All-optical synthesis chip for large-scale intelligent semantic vision generation," has been published in the prestigious international academic journal Science. Shanghai Jiao Tong University is listed as the first author and corresponding author affiliation, with Assistant Professor Chen Yitong serving as the first author and corresponding author.

Research Background

With the rapid evolution of deep neural networks and large-scale generative models, AI is transforming the world at an unprecedented pace. However, the explosive growth in the scale of generative models has led to surging demands for computing power and energy consumption, creating an increasingly critical and urgent gap compared to the performance growth rate of traditional chip architectures.

To overcome the bottlenecks in computing power and energy efficiency, novel architectures such as optical computing have attracted widespread attention. However, traditional all-optical computing chips are primarily limited to small-scale classification tasks, while optoelectronic cascading or multiplexing severely compromises the speed advantages of optical computing. Therefore, "how to enable next-generation optical computing chips to run complex generative models" has become a globally recognized challenge in the field of intelligent computing.Research Results

The research team has proposed, for the first time, the all-optical large-scale semantic generation chip, LightGen. This also marks the world's first implementation of a large-scale all-optical generative AI chip. It simultaneously addresses several widely recognized bottlenecks on a single chip, including the integration of millions of optical neurons, all-optical dimension transformation, and training algorithms for optical chips without ground-truth supervision.

The paper experimentally validates LightGen's performance in various large-scale generative tasks, including high-resolution (≥512×512) image semantic generation, 3D generation (NeRF), high-definition video generation with semantic control, denoising, and local and global feature transfer. Rather than relying on electrical assistance for generation, the all-optical chip fully accomplishes the end-to-end process: inputting an image, understanding semantics, manipulating semantics, and generating entirely new media data. In essence, it enables light to "understand" and "cognize" semantics.

Furthermore, LightGen adopts extremely stringent criteria for evaluating computing power. While achieving generative quality comparable to state-of-the-art electronic neural networks running on electrical chips (such as Stable Diffusion, NeRF, and Style Injection Diffusion), it directly measures significant reductions in end-to-end system latency and energy consumption. Actual measurements show that even with relatively outdated input devices, LightGen achieves improvements of over two orders of magnitude in computing power and energy efficiency compared to leading digital chips. If advanced devices are employed where signal input frequency is no longer a bottleneck, LightGen theoretically offers a performance leap of up to seven orders of magnitude in computing power and eight orders of magnitude in energy efficiency. This not only directly demonstrates the substantial gains in computing power and energy efficiency achievable by replacing current top-tier chips without compromising performance but also underscores the significance of overcoming key challenges—such as large-scale integration, all-optical dimension transformation, and ground-truth-free optical field training—to realize large-scale generative networks on all-optical chips.

The paper has also been selected by Science as a highlight article for featured coverage. The paper notes that as generative AI is increasingly integrated into production and daily life, it is imperative to develop chips capable of directly executing tasks required in the real world—particularly for large-scale generative models, which are highly sensitive to end-to-end latency and energy consumption. Toward this goal, LightGen opens a new pathway for next-generation computing chips to genuinely advance frontier artificial intelligence and provides a new research direction for exploring faster and more energy-efficient generative intelligent computing.