Microsoft researcher builds a working neural network out of goats in Age of Empires II to critique AI science

Editor's take

A Microsoft researcher has demonstrated a functional neural network constructed entirely within the Age of Empires II map editor, utilizing in-game units and terrain features.

This novel approach serves as a critique of the current AI research landscape, highlighting the potential for superficial achievements to be presented as significant breakthroughs. By simulating complex AI concepts with readily available, albeit unconventional, tools, the demonstration questions the rigor and reproducibility of certain published findings, particularly in light of the researcher's prior analysis of 315 AI papers where over half were found to lack sufficient detail.

The implications extend to how AI progress is measured and communicated within the academic and industry spheres. Future developments will likely involve increased scrutiny of experimental setups and a push for more transparent and robust validation methods, potentially leading to a recalibration of what constitutes impactful AI research beyond flashy demonstrations.