Gene

lamindb provides access to the following public gene ontologies through bionty:

  1. Ensembl

  2. NCBI Gene

Here we show how to access and search gene ontologies to standardize new data.

# pip install 'lamindb[bionty]'
!lamin init --storage ./test-public-ontologies --schema bionty
 initialized lamindb: testuser1/test-public-ontologies
import bionty as bt
import pandas as pd

PublicOntology objects

Let us create a public ontology accessor with public(), which chooses a default public ontology source from Source. It’s a PublicOntology object, which you can think about as a public registry:

public = bt.Gene.public(organism="human")
public
 connected lamindb: testuser1/test-public-ontologies
PublicOntology
Entity: Gene
Organism: human
Source: ensembl, release-112
#terms: 75829

As for registries, you can export the ontology as a DataFrame:

df = public.df()
df.head()
ensembl_gene_id symbol ncbi_gene_id biotype description synonyms
0 ENSG00000000003 TSPAN6 7105 protein_coding tetraspanin 6 TSPAN-6|T245|TM4SF6
1 ENSG00000000005 TNMD 64102 protein_coding tenomodulin TEM|MYODULIN|CHM1L|TENDIN|BRICD4
2 ENSG00000000419 DPM1 8813 protein_coding dolichyl-phosphate mannosyltransferase subunit... CDGIE|MPDS
3 ENSG00000000457 SCYL3 57147 protein_coding SCY1 like pseudokinase 3 PACE1|PACE-1
4 ENSG00000000460 FIRRM 55732 protein_coding FIGNL1 interacting regulator of recombination ... C1ORF112|FLJ10706|APOLO1|FLIP|MEICA1

Unlike registries, you can also export it as a Pronto object via public.ontology.

Look up terms

As for registries, terms can be looked up with auto-complete:

lookup = public.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.tcf7
Gene(ensembl_gene_id='ENSG00000081059', symbol='TCF7', ncbi_gene_id='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1')

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["TCF7"]
Gene(ensembl_gene_id='ENSG00000081059', symbol='TCF7', ncbi_gene_id='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1')

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = public.lookup(public.ncbi_gene_id)

If multiple entries are matched, they are returned as a list:

lookup.bt_100126572
Gene(ensembl_gene_id='ENSG00000203733', symbol='GJE1', ncbi_gene_id='100126572', biotype='protein_coding', description='gap junction protein epsilon 1 ', synonyms='CX23')

Search terms

Search behaves in the same way as it does for registries:

public.search("TP53").head(3)
ensembl_gene_id symbol ncbi_gene_id biotype description synonyms
8175 ENSG00000141510 TP53 7157 protein_coding tumor protein p53 LFS1|P53
15311 ENSG00000182165 TP53TG1 None lncRNA TP53 target 1 TP53AP1|H_RG012D21.9|LINC00096|TP53LC12
15671 ENSG00000183632 TP53TG3 24150 protein_coding TP53 target 3 P53TG3|TP53TG3A

By default, search also covers synonyms and all other fields containing strings:

public.search("PDL1").head(3)
ensembl_gene_id symbol ncbi_gene_id biotype description synonyms
5053 ENSG00000120217 CD274 29126 protein_coding CD274 molecule PD-L1|PDCD1LG1|B7H1|PDL1|B7-H1|B7-H
567 ENSG00000040275 SPDL1 54908 protein_coding spindle apparatus coiled-coil protein 1 CCDC99|HSPINDLY|FLJ20364
31380 ENSG00000229570 GAPDHP58 None processed_pseudogene glyceraldehyde 3 phosphate dehydrogenase pseud... GAPDHL1|GAPDL1

You can turn search only in symbols by passing field="symbol":

public.search("PDL1", field="symbol").head(3)
ensembl_gene_id symbol ncbi_gene_id biotype description synonyms
567 ENSG00000040275 SPDL1 54908 protein_coding spindle apparatus coiled-coil protein 1 CCDC99|HSPINDLY|FLJ20364

Search specific field (by default, search is done on all fields containing strings):

public.search("tumor protein p53", field=public.description).head()
ensembl_gene_id symbol ncbi_gene_id biotype description synonyms
1014 ENSG00000067369 TP53BP1 7158 protein_coding tumor protein p53 binding protein 1 53BP1|TDRD30|P202
1445 ENSG00000078804 TP53INP2 58476 protein_coding tumor protein p53 inducible nuclear protein 2 PINH|FLJ21759|DJ1181N3.1|DKFZP434O0827|FLJ2350...
4396 ENSG00000115129 TP53I3 9540 protein_coding tumor protein p53 inducible protein 3 PIG3
5088 ENSG00000120471 TP53AIP1 63970 protein_coding tumor protein p53 regulated apoptosis inducing... P53AIP1
8175 ENSG00000141510 TP53 7157 protein_coding tumor protein p53 LFS1|P53

Standardize gene identifiers

Let us generate a DataFrame that stores a number of gene identifiers, some of which corrupted:

data = {
    "gene symbol": ["A1CF", "A1BG", "FANCD1", "corrupted"],
    "ncbi id": ["29974", "1", "5133", "corrupted"],
    "ensembl_gene_id": [
        "ENSG00000148584",
        "ENSG00000121410",
        "ENSG00000188389",
        "ENSGcorrupted",
    ],
}
df_orig = pd.DataFrame(data).set_index("ensembl_gene_id")
df_orig
gene symbol ncbi id
ensembl_gene_id
ENSG00000148584 A1CF 29974
ENSG00000121410 A1BG 1
ENSG00000188389 FANCD1 5133
ENSGcorrupted corrupted corrupted

First we can check whether any of our values are validated against the ontology reference:

validated = public.validate(df_orig.index, public.ensembl_gene_id)
df_orig.index[~validated]
! 1 unique term (25.00%) is not validated: 'ENSGcorrupted'
Index(['ENSGcorrupted'], dtype='object', name='ensembl_gene_id')

Next, we validate which symbols are mappable against the ontology:

# based on NCBI gene ID
public.validate(df_orig["ncbi id"], public.ncbi_gene_id)
! 1 unique term (25.00%) is not validated: 'corrupted'
array([ True,  True,  True, False])
# based on Gene symbols
validated_symbols = public.validate(df_orig["gene symbol"], public.symbol)
df_orig["gene symbol"][~validated_symbols]
! 2 unique terms (50.00%) are not validated: 'FANCD1', 'corrupted'
ensembl_gene_id
ENSG00000188389       FANCD1
ENSGcorrupted      corrupted
Name: gene symbol, dtype: object

Here, 2 of the gene symbols are not validated. Inspect why:

public.inspect(df_orig["gene symbol"], public.symbol);
! 2 unique terms (50.00%) are not validated for symbol: 'FANCD1', 'corrupted'
   detected 1 unique terms with synonym: FANCD1
→  standardize terms via .standardize()

Logging suggests to use .standardize():

mapped_symbol_synonyms = public.standardize(df_orig["gene symbol"])
mapped_symbol_synonyms
['A1CF', 'A1BG', 'BRCA2', 'corrupted']

Optionally, you can return a mapper in the form of {synonym1: standardized_name1, ...}:

public.standardize(df_orig["gene symbol"], return_mapper=True)
{'FANCD1': 'BRCA2'}

We can use the standardized symbols as the new standardized index:

df_curated = df_orig.reset_index()
df_curated.index = mapped_symbol_synonyms
df_curated
ensembl_gene_id gene symbol ncbi id
A1CF ENSG00000148584 A1CF 29974
A1BG ENSG00000121410 A1BG 1
BRCA2 ENSG00000188389 FANCD1 5133
corrupted ENSGcorrupted corrupted corrupted

You can convert identifiers by passing return_field to standardize():

public.standardize(
    df_curated.index,
    field=public.symbol,
    return_field=public.ensembl_gene_id,
)
['ENSG00000148584', 'ENSG00000121410', 'ENSG00000139618', 'corrupted']

And return mappable identifiers as a dict:

public.standardize(
    df_curated.index,
    field=public.symbol,
    return_field=public.ensembl_gene_id,
    return_mapper=True,
)
{'A1BG': 'ENSG00000121410',
 'BRCA2': 'ENSG00000139618',
 'A1CF': 'ENSG00000148584'}

Ontology source versions

For any given entity, we can choose from a number of versions:

bt.Source.filter(entity="bionty.Gene").df()
Hide code cell output
uid entity organism name in_db currently_used description url md5 source_website space_id dataframe_artifact_id version run_id created_at created_by_id _aux _branch_code
id
11 4UGNz3fr bionty.Gene human ensembl False True Ensembl s3://bionty-assets/df_human__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
12 1HoNjuZL bionty.Gene human ensembl False False Ensembl s3://bionty-assets/df_human__ensembl__release-... None https://www.ensembl.org 1 None release-111 None 2025-03-10 13:25:14.948000+00:00 1 None 1
13 5dmX950R bionty.Gene human ensembl False False Ensembl s3://bionty-assets/df_human__ensembl__release-... None https://www.ensembl.org 1 None release-110 None 2025-03-10 13:25:14.948000+00:00 1 None 1
14 404rkf5M bionty.Gene human ensembl False False Ensembl s3://bionty-assets/human_ensembl_release-109_G... None https://www.ensembl.org 1 None release-109 None 2025-03-10 13:25:14.948000+00:00 1 None 1
15 4r4fvV0S bionty.Gene mouse ensembl False True Ensembl s3://bionty-assets/df_mouse__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
16 5yZh7H6v bionty.Gene mouse ensembl False False Ensembl s3://bionty-assets/df_mouse__ensembl__release-... None https://www.ensembl.org 1 None release-111 None 2025-03-10 13:25:14.948000+00:00 1 None 1
17 34Tj65wq bionty.Gene mouse ensembl False False Ensembl s3://bionty-assets/df_mouse__ensembl__release-... None https://www.ensembl.org 1 None release-110 None 2025-03-10 13:25:14.948000+00:00 1 None 1
18 PGj9lGUN bionty.Gene mouse ensembl False False Ensembl s3://bionty-assets/mouse_ensembl_release-109_G... None https://www.ensembl.org 1 None release-109 None 2025-03-10 13:25:14.948000+00:00 1 None 1
19 4RPA3Re0 bionty.Gene saccharomyces cerevisiae ensembl False True Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
20 4YyqUC3S bionty.Gene saccharomyces cerevisiae ensembl False False Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... None https://www.ensembl.org 1 None release-111 None 2025-03-10 13:25:14.948000+00:00 1 None 1
21 772aK35Y bionty.Gene saccharomyces cerevisiae ensembl False False Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... None https://www.ensembl.org 1 None release-110 None 2025-03-10 13:25:14.948000+00:00 1 None 1
# only lists the sources that are currently used
bt.Source.filter(entity="bionty.Gene", currently_used=True).df()
uid entity organism name in_db currently_used description url md5 source_website space_id dataframe_artifact_id version run_id created_at created_by_id _aux _branch_code
id
11 4UGNz3fr bionty.Gene human ensembl False True Ensembl s3://bionty-assets/df_human__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
15 4r4fvV0S bionty.Gene mouse ensembl False True Ensembl s3://bionty-assets/df_mouse__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
19 4RPA3Re0 bionty.Gene saccharomyces cerevisiae ensembl False True Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1

When instantiating a Bionty object, we can choose a source or version:

source = bt.Source.get(name="ensembl", version="release-112", organism="human")
public = bt.Gene.public(source=source)
public
PublicOntology
Entity: Gene
Organism: human
Source: ensembl, release-112
#terms: 75829

The currently used ontologies can be displayed using:

bt.Source.filter(currently_used=True).df()
Hide code cell output
uid entity organism name in_db currently_used description url md5 source_website space_id dataframe_artifact_id version run_id created_at created_by_id _aux _branch_code
id
1 33TUF039 bionty.Organism vertebrates ensembl False True Ensembl https://ftp.ensembl.org/pub/release-112/specie... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
6 6bbVUTCS bionty.Organism bacteria ensembl False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... None https://www.ensembl.org 1 None release-57 None 2025-03-10 13:25:14.948000+00:00 1 None 1
7 6s9nV6xh bionty.Organism fungi ensembl False True Ensembl http://ftp.ensemblgenomes.org/pub/fungi/releas... None https://www.ensembl.org 1 None release-57 None 2025-03-10 13:25:14.948000+00:00 1 None 1
8 2PmTrc8x bionty.Organism metazoa ensembl False True Ensembl http://ftp.ensemblgenomes.org/pub/metazoa/rele... None https://www.ensembl.org 1 None release-57 None 2025-03-10 13:25:14.948000+00:00 1 None 1
9 7GPHh16S bionty.Organism plants ensembl False True Ensembl https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... None https://www.ensembl.org 1 None release-57 None 2025-03-10 13:25:14.948000+00:00 1 None 1
10 4tsksCMX bionty.Organism all ncbitaxon False True NCBItaxon Ontology s3://bionty-assets/df_all__ncbitaxon__2023-06-... None https://github.com/obophenotype/ncbitaxon 1 None 2023-06-20 None 2025-03-10 13:25:14.948000+00:00 1 None 1
11 4UGNz3fr bionty.Gene human ensembl False True Ensembl s3://bionty-assets/df_human__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
15 4r4fvV0S bionty.Gene mouse ensembl False True Ensembl s3://bionty-assets/df_mouse__ensembl__release-... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
19 4RPA3Re0 bionty.Gene saccharomyces cerevisiae ensembl False True Ensembl s3://bionty-assets/df_saccharomyces cerevisiae... None https://www.ensembl.org 1 None release-112 None 2025-03-10 13:25:14.948000+00:00 1 None 1
22 3EYyGRYN bionty.Protein human uniprot False True Uniprot s3://bionty-assets/df_human__uniprot__2024-03_... None https://www.uniprot.org 1 None 2024-03 None 2025-03-10 13:25:14.948000+00:00 1 None 1
25 01RWXN2V bionty.Protein mouse uniprot False True Uniprot s3://bionty-assets/df_mouse__uniprot__2024-03_... None https://www.uniprot.org 1 None 2024-03 None 2025-03-10 13:25:14.948000+00:00 1 None 1
28 3kDh8qAX bionty.CellMarker human cellmarker False True CellMarker s3://bionty-assets/human_cellmarker_2.0_CellMa... None http://bio-bigdata.hrbmu.edu.cn/CellMarker 1 None 2.0 None 2025-03-10 13:25:14.948000+00:00 1 None 1
29 7bV5uJo3 bionty.CellMarker mouse cellmarker False True CellMarker s3://bionty-assets/mouse_cellmarker_2.0_CellMa... None http://bio-bigdata.hrbmu.edu.cn/CellMarker 1 None 2.0 None 2025-03-10 13:25:14.948000+00:00 1 None 1
30 6LyRtvz8 bionty.CellLine all clo False True Cell Line Ontology https://data.bioontology.org/ontologies/CLO/su... None https://bioportal.bioontology.org/ontologies/CLO 1 None 2022-03-21 None 2025-03-10 13:25:14.948000+00:00 1 None 1
32 3Uw2Va7a bionty.CellType all cl False True Cell Ontology http://purl.obolibrary.org/obo/cl/releases/202... None https://obophenotype.github.io/cell-ontology 1 None 2024-08-16 None 2025-03-10 13:25:14.948000+00:00 1 None 1
41 MUtAGdL4 bionty.Tissue all uberon False True Uberon multi-species anatomy ontology http://purl.obolibrary.org/obo/uberon/releases... None http://obophenotype.github.io/uberon 1 None 2024-08-07 None 2025-03-10 13:25:14.948000+00:00 1 None 1
50 4a3ejKuf bionty.Disease all mondo False True Mondo Disease Ontology http://purl.obolibrary.org/obo/mondo/releases/... None https://mondo.monarchinitiative.org 1 None 2024-08-06 None 2025-03-10 13:25:14.948000+00:00 1 None 1
59 4kswnHVF bionty.Disease human doid False True Human Disease Ontology http://purl.obolibrary.org/obo/doid/releases/2... None https://disease-ontology.org 1 None 2024-05-29 None 2025-03-10 13:25:14.952000+00:00 1 None 1
67 2a1HvjdB bionty.ExperimentalFactor all efo False True The Experimental Factor Ontology http://www.ebi.ac.uk/efo/releases/v3.70.0/efo.owl None https://bioportal.bioontology.org/ontologies/EFO 1 None 3.70.0 None 2025-03-10 13:25:14.952000+00:00 1 None 1
75 48fBFLmn bionty.Phenotype human hp False True Human Phenotype Ontology https://github.com/obophenotype/human-phenotyp... None https://hpo.jax.org 1 None 2024-04-26 None 2025-03-10 13:25:14.952000+00:00 1 None 1
80 4t7QibxO bionty.Phenotype mammalian mp False True Mammalian Phenotype Ontology https://github.com/mgijax/mammalian-phenotype-... None https://github.com/mgijax/mammalian-phenotype-... 1 None 2024-06-18 None 2025-03-10 13:25:14.952000+00:00 1 None 1
83 sqPX2b7b bionty.Phenotype zebrafish zp False True Zebrafish Phenotype Ontology https://github.com/obophenotype/zebrafish-phen... None https://github.com/obophenotype/zebrafish-phen... 1 None 2024-04-18 None 2025-03-10 13:25:14.952000+00:00 1 None 1
87 6S4qkDx1 bionty.Phenotype all pato False True Phenotype And Trait Ontology http://purl.obolibrary.org/obo/pato/releases/2... None https://github.com/pato-ontology/pato 1 None 2024-03-28 None 2025-03-10 13:25:14.952000+00:00 1 None 1
89 7Ent3V2y bionty.Pathway all go False True Gene Ontology https://data.bioontology.org/ontologies/GO/sub... None http://geneontology.org 1 None 2024-06-17 None 2025-03-10 13:25:14.952000+00:00 1 None 1
94 3rm9aOzL BFXPipeline all lamin False True Bioinformatics Pipeline s3://bionty-assets/df_all__lamin__1.0.0__BFXpi... None https://lamin.ai 1 None 1.0.0 None 2025-03-10 13:25:14.952000+00:00 1 None 1
95 ugaIoIlj Drug all dron False True Drug Ontology https://data.bioontology.org/ontologies/DRON/s... None https://bioportal.bioontology.org/ontologies/DRON 1 None 2024-08-05 None 2025-03-10 13:25:14.952000+00:00 1 None 1
99 1GbFkOdz bionty.DevelopmentalStage human hsapdv False True Human Developmental Stages https://github.com/obophenotype/developmental-... None https://github.com/obophenotype/developmental-... 1 None 2024-05-28 None 2025-03-10 13:25:14.952000+00:00 1 None 1
101 10va5JSt bionty.DevelopmentalStage mouse mmusdv False True Mouse Developmental Stages https://github.com/obophenotype/developmental-... None https://github.com/obophenotype/developmental-... 1 None 2024-05-28 None 2025-03-10 13:25:14.952000+00:00 1 None 1
103 MJRqduf9 bionty.Ethnicity human hancestro False True Human Ancestry Ontology https://github.com/EBISPOT/hancestro/raw/3.0/h... None https://github.com/EBISPOT/hancestro 1 None 3.0 None 2025-03-10 13:25:14.952000+00:00 1 None 1
104 5JnVODh4 BioSample all ncbi False True NCBI BioSample attributes s3://bionty-assets/df_all__ncbi__2023-09__BioS... None https://www.ncbi.nlm.nih.gov/biosample/docs/at... 1 None 2023-09 None 2025-03-10 13:25:14.952000+00:00 1 None 1