Knowledge Center
Where the data comes from
The NIAID Data Ecosystem Discovery Portal harvests metadata from a number of different data sources to make it easier to find allergic or infectious and immune-mediated disease (IID) datasets.
What is metadata?#
Metadata is data about resources — who collected the data, how it was collected, what the resource contains, etc. For something like a genome sequence, the data would be the actual sequence of nucleotides, while the metadata is the author of the data, the date the data was modified, the measurementTechniques used to collect the data, the healthCondition at the focus of dataset (like COVID-19, asthma, or autoimmune diseases), and more. Using the Discovery Portal, users can search these metadata to find datasets of interest.
Where does the data come from?#
The Discovery Portal collects metadata from sources that are related to allergic, infectious and immune-mediated disease (IID). Some sources included in the Discovery Portal are also general biological data sources or generalist repositories. The breakdown of the number of resources by source is found on the Discovery Portal homepage. The Sources page provides more details about each source and the last time the metadata was harvested. The sources used by the Discovery Portal are also constantly expanding.
How are the sources chosen?#
Interviews with researchers about their needs informed the selection of the initial sources included in the Discovery Portal. As the Discovery Portal is continuously growing, the team is always looking to add other sources to the search platform.
📘 Want to suggest a new source?
How often is metadata harvested?#
Currently, metadata is harvested every quarter. The Sources page lists the last time the metadata was collected from the source.
Is the metadata manipulated?#
Some of the metadata provided by the sources is cleaned so it is more standardized and easily searchable. Mostly, this involves standardizing the names of metadata fields (variables) so they are consistent between sources (read more about data schemas). These changes are tracked on the schema tab of the Sources page:
You can also read more about metadata standardization here.
📘 Accessing the standardized metadata
The standardized metadata can be freely accessed using the Discovery Portal's open metadata API
Last updated on