· 2024
Rewrite the Stars
- 513citations
- 2024year
Short summary
This study reveals that the 'star operation' (element-wise multiplication) in neural networks can map inputs into high-dimensional, non-linear feature spaces without increasing network width, similar to kernel tricks.
AI-generated from the title and abstract; the full text is not read.
Key points
- The star operation (element-wise multiplication) implicitly maps inputs to high-dimensional, non-linear feature spaces.
- This mapping occurs without increasing the network's width, offering computational efficiency.
- A prototype network, StarNet, demonstrates impressive performance and low latency.
- The operation's potential is compared to kernel tricks used in machine learning.
AI-generated from the title and abstract; the full text is not read.
Abstract
Recent studies have drawn attention to the untapped potential of the “star operation” (element-wise multiplication) in network design. While intuitive explanations abound, the foundational rationale behind its application remains largely unexplored. Our study attempts to reveal the star operation's ability of mapping inputs into high-dimensional, non-linear feature spaces-akin to kernel tricks-without widening the network. We further introduce StarNet, a simple yet pow-erful prototype, demonstrating impressive performance and low latency under compact network structure and efficient budget. Like stars in the sky, the star operation appears unremarkable but holds a vast universe of potential. Our work encourages further exploration across tasks, with codes available at https://github.com/ma-xu/Rewrite-the-Stars.
The authors' abstract, as published at the source., 2024 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Archeology (Arts and Humanities)
ArcheologyArts and Humanities