NeuroMorpho.Org agile release pipeline enables faster data sharing

Abstract

NeuroMorpho.Org is a curated open-access repository of neural morphology. Over 1000 laboratories worldwide contributed more than 250,000 three-dimensional reconstructions of neurons and glia that can be browsed by species, brain regions, and cell types. These data are re-used for many scientific applications, leading to new discoveries every week. NeuroMorpho.Org is continuously growing based on a 3-step pipeline: literature mining, data processing, and metadata annotation. Earlier in the project, we released multiple datasets together yearly, but this process caused an average delay of several months in making the data public. Moreover, increased data sharing in the neuroscience community created the need for automatic processing to release more reconstructions in less time. We have progressively automated the NeuroMorpho.Org processing pipeline and modernized the database to enable agile releases of individual datasets as soon as they are ready. Here we analyze the effects of these pipeline improvements. The overall processing time decreased by 35%; 82% of the datasets are now released in less than 3 months compared to 35% prior to agile releases. Furthermore, the average processing time per reconstruction dropped from 56 hours to 8.7 hours. These findings demonstrate that agile releasing enables faster data sharing, thus benefiting both authors and users.