Minerals are solid state nuclear track detectors - nuclear recoils in a mineral leave latent damage to the crystal structure. Depending on the mineral and its temperature, the damage features are retained in the material from minutes to timescales much larger than the age of the Solar System. The damage features from the fission fragments left by spontaneous fission of heavy unstable isotopes have long been used for fission track dating of geological samples. Laboratory studies have demonstrated the readout of defects caused by nuclear recoils with energies as small as ~1 keV. Using natural minerals, one could use the damage features accumulated over geological timescales to measure astrophysical neutrino fluxes (from the Sun, supernovae, or cosmic rays interacting with the atmosphere) as well as search for Dark Matter. Research groups in Europe, Asia, and America have started developing microscopy techniques to read out the nanoscale damage features in crystals left by keV nuclear recoils. The research program towards the realization of such mineral detectors is highly interdisciplinary, combining geoscience, material science, applied and fundamental physics with techniques from quantum information and Artificial Intelligence. In this talk, I will highlight the scientific potential of Dark Matter searches with mineral detectors and briefly describe status and plans of the Mineral Detection of Neutrinos and Dark Matter (MDvDM) community.