![]() GMOD Resources
Demos Links to Related Projects
![]() GMOD is supported by a specific cooperative agreement from the USDA Agricultural Research Service, and by NIH grants co-funded from the National Human Genome Research Institute and the National Institute of General Medical Sciences.
|
CONTENTS
A. CREATING NEW DATABASES FROM SCRATCHThis section describes how to create new annotation databases from scratch.
A1. The GFF file formatGBrowse is based around the GFF file format, which stands for ``Gene Finding Format'' and was invented at the Sanger Centre. The GFF format is a flat tab-delimited file, each line of which corresponds to an annotation, or feature. Each line has nine columns and looks like this: Chr1 curated CDS 365647 365963 . + 1 Transcript "R119.7" The 9 columns are as follows:
The sequences used to establish the coordinate system for annotations can correspond to sequenced clones, clone fragments, contigs or super-contigs. In addition to a group ID, the GFF format allows annotations to have a group class. This makes sure that all groups are unique even if they happen to share the same name. For example, you can have a GenBank accession named AP001234 and a clone named AP001234 and distinguish between them by giving the first one a class of Accession and the second a class of Clone. You should use double-quotes around the group name or class if it contains white space.
A2. Creating a GFF tableThe first 8 fields of the GFF format are easy to understand. The group field is a challenge. It is used in three distinct ways:
1. Using the Group field for simple features For a simple feature that spans a single continuous range, choose a name and class for the object and give it a line in the GFF file that refers to its start and stop positions. Chr3 giemsa heterochromatin 4500000 6000000 . . . Band 3q12.1 2. Using the Group field to group features that belong together For a group of features that belong together, such as the exons in a transcript, choose a name and class for the object. Give each segment a separate line in the GFF file but use the same name for each line. For example: IV curated exon 5506900 5506996 . + . Transcript B0273.1 IV curated exon 5506026 5506382 . + . Transcript B0273.1 IV curated exon 5506558 5506660 . + . Transcript B0273.1 IV curated exon 5506738 5506852 . + . Transcript B0273.1 These four lines refer to a biological object of class ``Transcript'' and name B0273.1. Each of its parts uses the method ``exon'', source ``curated''. Once loaded, the user will be able to search the genome for this object by asking the browser to retrieve ``Transcript:B0273.1''. The browser can also be configured to allow the Transcript: prefix to be omitted. You can extend the idiom for objects that have heterogeneous parts, such as a transcript that has 5' and 3' UTRs IV curated mRNA 5506800 5508917 . + . Transcript B0273.1; Note "Zn-Finger" IV curated 5'UTR 5506800 5508999 . + . Transcript B0273.1 IV curated exon 5506900 5506996 . + . Transcript B0273.1 IV curated exon 5506026 5506382 . + . Transcript B0273.1 IV curated exon 5506558 5506660 . + . Transcript B0273.1 IV curated exon 5506738 5506852 . + . Transcript B0273.1 IV curated 3'UTR 5506852 5508917 . + . Transcript B0273.1 In this example, there is a single feature with method ``mRNA'' that spans the entire range. It is grouped with subparts of type 5'UTR, 3'UTR and exon. They are all grouped together into a Transcript named B0273.1. Furthermore the mRNA feature has a note attached to it. *NOTE* The subparts of a feature are in absolute (chromosomal or contig) coordinates. It is not currently possible to define a feature in absolute coordinates and then to load its subparts using coordinates that are relative to the start of the feature. Some annotations do not need to be individually named. For example, it is probably not useful to assign a unique name to each ALU repeat in a vertebrate genome. For these, just leave the Group field empty. 3. Using the Group field to add a note The group field can be used to add one or more notes to an annotation. To do this, place a semicolon after the group name and add a Note field: Chr3 giemsa heterochromatin 4500000 6000000 . . . Band 3q12.1 ; Note "Marfan's syndrome" You can add multiple Notes. Just separate them by semicolons: Band 3q12.1 ; Note "Marfan's syndrome" ; Note "dystrophic dysplasia" The Note should come AFTER the group type and name.
A3. Identifying the reference sequenceEach reference sequence in the GFF table must itself have an entry. This is necessary so that the length of the reference sequence is known. For example, if ``Chr1'' is used as a reference sequence, then the GFF file should have an entry for it similar to this one: Chr1 assembly chromosome 1 14972282 . + . Sequence Chr1 This indicates that the reference sequence named ``Chr1'' has length 14972282 bp, method ``chromosome'' and source ``assembly''. In addition, as indicated by the group field, Chr1 has class ``Sequence'' and name ``Chr''. It is suggested that you use ``Sequence'' as the class name for all reference sequences, since this is the default class used by the Bio::DB::GFF module when no more specific class is requested. If you use a different class name, then be sure to indicate that fact with the ``reference class'' option (see below).
A4. Sequence alignmentsThere are several cases in which an annotation indicates the relationship between two sequences. One common one is a similarity hit, where the annotation indicates an alignment. A second common case is a map assembly, in which the annotation indicates that a portion of a larger sequence is built up from one or more smaller ones. Both cases are indicated by using the Target tag in the group field. For example, a typical similarity hit will look like this: Chr1 BLASTX similarity 76953 77108 132 + 0 Target Protein:SW:ABL_DROME 493 544 Here, the group field contains the Target tag, followed by an identifier for the biological object. The GFF format uses the notation Class:Name for the biological object, and even though this is stylistically inconsistent, that's the way it's done. The object identifier is followed by two integers indicating the start and stop of the alignment on the target sequence. Unlike the main start and stop columns, it is possible for the target start to be greater than the target end. The previous example indicates that the the section of Chr1 from 76,953 to 77,108 aligns to the protein SW:ABL_DROME starting at position 493 and extending to position 544. A similar notation is used for sequence assembly information as shown in this example: Chr1 assembly Link 10922906 11177731 . . . Target Sequence:LINK_H06O01 1 254826 LINK_H06O01 assembly Cosmid 32386 64122 . . . Target Sequence:F49B2 6 31742 This indicates that the region between bases 10922906 and 11177731 of Chr1 are composed of LINK_H06O01 from bp 1 to bp 254826. The region of LINK_H0601 between 32386 and 64122 is, in turn, composed of the bases 5 to 31742 of cosmid F49B2.
A6. Loading the GFF file into the databaseUse the BioPerl script utilities bulk_load_gff.pl, load_gff.pl or (if you are brave) fast_load_gff.pl to load the GFF file into the database. For example, if your database is a MySQL database on the local host named ``dicty'', you can load it into an empty database using bulk_load_gff.pl like this: bulk_load_gff.pl -c -d dicty my_data.gff To update existing databases, use either load_gff.pl or fast_load_gff.pl. The latter is somewhat experimental, so use with care.
A5. AggregatorsThe Bio::DB::GFF module has a feature known as ``aggregators''. These are small software packages that recognize certain common feature types and convert them into complex biological objects. These aggregators make it possible to develop intelligent graphical representations of annotations, such as a gene that draws confirmed exons differently from predicted ones. An aggregator typically creates a new composite feature with a different method than any of its components. For example, the standard ``alignment'' aggregator takes multiple alignments of method ``similarity'', groups them by their name, and returns a single feature of method ``alignment''. The various aggregators are described in detail in the Bio::DB::GFF manual page. It is easy to write new aggregators, and also possible to define aggregators on the fly in the gbrowse configuration file. It is suggested that you use the sample GFF files from the yeast, drosophila and C. elegans projects to see what methods to use to achieve the desired results.
B. ADDING A NEW DATABASE TO THE BROWSEREach data source has a corresponding configuration file in the directory gbrowse.conf. Once you've created and loaded a new database, you should make a copy of one of the existing configuration files and modify it to meet your needs. The name of the new configuration file must follow the form: sourcename.conf where ``sourcename'' is a short word that describes the data source. You can use this name to select the data source when linking to the browser. Just construct a URL that uses ``sourcename'' as a virtual directory under cgi-bin/gbrowse: http://your.site.org/cgi-bin/gbrowse/sourcename/ (Note: If you don't add the slash at the end, gbrowse will automatically do it for you, since the terminal slash is needed to work around an apparent bug in MSIE's cookie handling.) It is suggested that you use the same name as the database, although this isn't a requirement. (If no ``source='' argument is given, gbrowse picks the first configuration file that occurs alphabetically; you can control this by placing numbers in front of the configuration file, as in ``01.yeast.conf''.) The configuration file is divided into a number of sections, each one introduced by a [SECTION TITLE]. The [GENERAL] section contains settings that are applicable to the entire application. Other sections define tracks to display. I suggest that you begin with one of the example configuration files provided with the distribution and modify it to suit your needs.
B1. The [GENERAL] SectionThe [GENERAL] section consists of a series of name=value options. For example, the beginning of the yeast.conf sample configuration file looks like this:
[GENERAL]
description = S. cerevisiae (via SGD Nov 2001)
db_adaptor = Bio::DB::GFF
db_args = -adaptor dbi::mysql
-dsn dbi:mysql:database=yeast;host=localhost
aggregators = transcript alignment
user =
passwd =
Each option is a single word or phrase, usually in lower case. This is followed by an equals sign and the value of the option. You can add whitespace around the equals sign in order to increase readability. If a value is very long, you can continue it on additional lines provided that you put a tab or other whitespace on the continuation lines. For example:
description = S. cerevisiae annotations via SGD Nov 2001, and
converted using the process_sgd.pl script
Any lines that begin with a pound sign (#) are considered comments and ignored. During this discussion, you might want to follow along with one of the example configuration files. The following [GENERAL] options are recognized:
B2. The [TRACK DEFAULTS] sectionThe track defaults section specifies default values for each track. The following common options are recognized:
glyph
height
bgcolor
fgcolor
fontcolor
font2color
strand_arrow
These options control the default graphical settings for any annotation types that are not explicitly specified. See the section below on controlling the settings. Any of the options allowed in the [track] sections described below are allowed here.
B3. Track SectionsAny other [Section] in the configuration file is treated as a declaration of a track. The order of track sections will become the default order of tracks on the display (the user can change this later). Here is a typical track declaration from yeast.conf: [Genes] feature = gene:sgd glyph = generic bgcolor = yellow forwardcolor = yellow reversecolor = turquoise strand_arrow = 1 height = 6 description = 1 key = Named gene The track is named ``Genes''. You may use a short mnemonic if you prefer. As in the general configuration section, the track declaration contains multiple name=value option pairs. Valid options are as follows:
A large number of glyph-specific options are also recognized. These are described in the next section.
B4. Glyphs and Glyph OptionsA large variety of glyphs are available, and more are being added as the Bio::Graphics module grows. A list of the common glyphs and their options is provided by the GBrowse itself. Click on the ``[Help]'' link in the section labeled ``Upload your own annotations''. This page also lists the valid foreground and background colors. Most of the glyphs are found in the BioPerl distribution, but a few are distributed directly with GBrowse. The most popular glyph types are: Glyph Description ----- -----------
generic a rectangle
allele_tower allele found at a SNP position
arrow an arrow
anchored_arrow a span with vertical bases |---------|. If one
or the other end of the feature is off-screen, the
base will be replaced by an arrow.
box another rectangle; doesn't show subparts of features
cds shows the reading frame of spliced transcripts; used
in conjunction with the "coding" aggregator.
diamond a point-like feature represented as a triangle
dna DNA and GC content
heterogeneous_segments a multi-segmented feature in which each segment can
have a distinctive color. For Jim Kent's WABA features,
this works with the waba_alignment aggregator.
idiogram this takes specially-formatted feature data and turns it
into an idiogram of a Giemsa-stained metaphase chromosome
processed_transcript multi-purpose representation of a spliced mRNA, including
positions of UTRs
segments a multi-segmented feature such as an alignment
span like anchored_arrow, except that the ends are
truncated at the edge of the panel, not turned
into an arrow
triangle a point-like feature represented as a diamond
transcript a gene model
transcript2 a slightly different representation of a gene model
translation 1-, 3- and 6-frame translations
wormbase_transcript yet another gene model that can show UTR segments
(for features that conform to the WormBase gene
schema). Used in conjunction with the
"wormbase_gene" aggregator.
xyplot histograms and line plots
A more definitive list of glyph options can be found in the Bio::Graphics manual pages. Consult the manual pages for the following modules: Glyph Manual Page ----- ----------- (common options for all) Bio::Graphics::Glyph allele_tower Bio::Graphics::Glyph::allele_tower arrow Bio::Graphics::Glyph::arrow anchored_arrow Bio::Graphics::Glyph::anchored_arrow box Bio::Graphics::Glyph::box cds Bio::Graphics::Glyph::cds crossbox Bio::Graphics::Glyph::crossbox diamond Bio::Graphics::Glyph::diamond dna Bio::Graphics::Glyph::dna dot Bio::Graphics::Glyph::dot ellipse Bio::Graphics::Glyph::ellipse extending_arrow Bio::Graphics::Glyph::extending_arrow generic Bio::Graphics::Glyph::generic graded_segments Bio::Graphics::Glyph::graded_segments heterogeneous_segments Bio::Graphics::Glyph::heterogeneous_segments idiogram Bio::Graphics::Glyph::idiogram line Bio::Graphics::Glyph::line primers Bio::Graphics::Glyph::primers processed_transcript Bio::Graphics::Glyph::processed_transcript rndrect Bio::Graphics::Glyph::rndrect ruler_arrow Bio::Graphics::Glyph::ruler_arrow segments Bio::Graphics::Glyph::segments span Bio::Graphics::Glyph::span toomany Bio::Graphics::Glyph::toomany transcript Bio::Graphics::Glyph::transcript transcript2 Bio::Graphics::Glyph::transcript2 translation Bio::Graphics::Glyph::translation triangle Bio::Graphics::Glyph::triangle wormbase_transcript Bio::Graphics::Glyph::wormbase_transcript xyplot Bio::Graphics::Glyph::xyplot The ``perldoc'' command is handy for reading the documentation from the Unix command line. For example: perldoc Bio::Graphics::Glyph::primers This will provide you with a summary of the options that apply to the ``primers'' glyph. In the manual pages, the glyph options are presented the way they are called from Perl. For example, the documentation will tell you to use the -connect_color option to set the color to use when drawing the line that connects the two inward pointing arrows in the primer pair glyph. This translates to the configuration file as an option named ``connect_color''. For example: [PCR Products] glyph = primer connect_color = blue When referring to colors, you can use a variety of color names such as ``blue'' and ``green''. To get the full list, cut and paste the following magic incantation into the command line: perl -MBio::Graphics::Panel -e 'print join "\n",Bio::Graphics::Panel->color_names' or see this URL: http://www.wormbase.org/db/seq/gbrowse?help=annotation Alternatively, you can use the #RRGGBB notation to specify the red, green and blue components of the color. Refer to any book on HTML for the details on using the notation.
B5. Adding features to the overviewYou can make any set of tracks appear in the overview by creating a stanza with a title of the format [<label>:overview], where <label> is any unique label of your choice. The format of the stanza is identical to the others, but the indicated track will appear in the overview rather than as an option in the detailed view. For example, this stanza adds to the overview a set of features of method ``gene'', source ``framework'': [framework:overview] feature = gene:framework label = 1 glyph = generic bgcolor = lavender height = 5 key = Mapped Genes
B6. Semantic ZoomingSometimes you will want to change the appearance of a track when the user has zoomed out or zoomed in beyond a certain level. To indicate this, create a set of ``length qualified'' stanzas of format [<label>:<zoom level>], where all stanzas share the same <label>, and <zoom level> indicates the minimum size of the region that the stanza will apply to. For example: [gene] feature = transcript:curated glyph = dna fgcolor = blue key = genes citation = example semantic zoom track [gene:500] feature = transcript:curated glyph = transcript2 [gene:100000] feature = transcript:curated glyph = arrow [gene:500000] feature = transcript:curated glyph = generic This series of stanzas says to use the ``transcript2'' glyph when the segment being displayed is 500 bp or longer, to use the ``arrow'' glyph when the segment being displayed is 100,000 bp or longer, and the ``generic'' glyph when the region being displayed is 500,000 bp or longer. For all other segment lengths (1 to 499 bp), the ordinary [gene] stanza will be consulted, and the ``dna'' glyph will be displayed. The bare [gene] stanza is used to set all but the ``feature'' options for the other stanzas. This means that the fgcolor, key and citation options are shared amongst all the [gene:XXXX] stanzas, but the ``feature'' option must be repeated. You can override any options in the length qualified stanzas. For example, if you want to change the color to red in when displaying genes on segments between 500 and 99,999 bp, you can modify the [gene:500] stanza as follows: [gene:500] feature = transcript:curated glyph = transcript2 fgcolor = red It is also possible to display different features at different zoom levels, although you should handle this potentially confusing feature with care!
B7. Computed OptionsSome options can be computed at run time by using Perl subroutines as their values. Currently this works with the values of the ``link'', ``title'', ``header'' and ``footer'' options, and any glyph-specific option that appears in a track section. You need to know the Perl programming language to take advantage of this. The general format of this type of option is:
option name = sub {
some perl code;
some more perl code;
even more perl code;
}
The value must begin with the sequence ``sub {'' in order to be recognized as a subroutine declaration. After this, you can have one or more lines of Perl code followed by a closing brace. Continuation lines must begin with whitespace. When the browser first encounters an option like this one, it will attempt to compile it into Perl runtime code. If successful, the compiled code will be stored for later use and invoked whenever the value of the option is needed. (Otherwise, an error message will appear in your server error log). For options of type ``footer'' and ``header'', the subroutine is passed no arguments. It is expected to produce some HTML and return it as a string value. For the ``link'' option, the subroutine will be called with a single argument consisting of a Bio::DB::GFF::Feature object. The subroutine should examine this object to determine what URL to link to and return the value of the URL as a string, or undef if no link should be made. See the manual page for Bio::DB::GFF::Feature for information on how to interrogate the feature object. For glyph-specific features, the subroutine will be called at run time
with the feature (a Bio::SeqI-complaint object) as the first argument.
The subroutine can examine this feature and determine what value to
return. The return value is treated as the value of the corresponding
option. For example, this bgcolor subroutine will call the feature's
bgcolor = sub {
my $feature = shift;
return "blue" if $feature->primary_tag eq 'exon';
return "orange";
}
More subroutine arguments are available to the glyph-specific subroutines. See the Bio::Graphics::Panel manual page for the full details. Named Subroutine References --------------------------- If you use a version of BioPerl after April 15, 2003, you can also use references to named subroutines as option arguments. To use named subroutines, add an init_code section to the [GENERAL] section of the configuration file. init_code should contain nothing but subroutine definitions and other initialization routines. For example:
init_code = sub score_color {
my $feature = shift;
if ($feature->score > 50) {
return 'red';
} else {
return 'green';
}
}
sub score_height {
my $feature = shift;
if ($feature->score > 50) {
return 10;
} else {
return 5;
}
}
Then simply refer to these subroutines using the \&name syntax:
[EST_ALIGNMENTS]
glyph = generic
bgcolor = \&score_color
height = \&score_height
You can declare global variables in the init_code subroutine if you use ``no strict 'vars';'' at the top of the section:
init_code = no strict 'vars';
$HEIGHT = 10;
sub score_height {
my $feature = shift;
$HEIGHT++;
if ($feature->score > 50) {
return $HEIGHT*2;
} else {
return $HEIGHT;
}
}
Due to the way the configuration file is parsed, there must be no empty lines in the init_code section. Either use comments to introduce white space, or ``use'' a .pm file to do anything fancy.
B8. Declaring New AggregatorsThe Bio::DB::GFF data model recognizes a single-level of ``grouping'' of features, but doesn't specify how to use the group information to correctly assemble the various individual components into a biological object. Aggregators are used to assemble this information. For example, let's say that you decide that your preferred ``transcript'' data model contains three subfeature types: a set of one or more features of method ``exon'', a single feature of method ``TSS'', and a single feature of method ``polyA''. Optionally, the data model could contain a single ``main subfeature'' that runs the length of the entire transcript. We might give this feature a method of ``primary_transc'' (for ``primary transcript.'') In a GFF file, a three-exon transcript might be represented as follows: Chr1 confirmed primary_transc 100 500 . + . Transcript "ABC.1" Chr1 confirmed TSS 100 100 . + . Transcript "ABC.1" Chr1 confirmed exon 100 200 . + . Transcript "ABC.1" Chr1 confirmed exon 250 300 . + . Transcript "ABC.1" Chr1 confirmed exon 400 500 . + . Transcript "ABC.1" Chr1 confirmed polyA 500 500 . + . Transcript "ABC.1" To aggregate this, you would like to create an aggregator named ``transcript'', whose ``main method'' is ``primary_transc'', and whose ``sub methods'' are ``TSS,'' ``exon,'' and ``polyA.'' The way to indicate this in the configuration file is to add a ``complex aggregator'' to the list of aggregators:
aggregator = transcript{TSS,exon,polyA/primary_transc}
The format of this value is ``aggregator_name{submethod1,submethod2,.../mainmethod}''. You can now use the name of the aggregator name as the argument of the ``feature'' option in a track section: [Transcripts] feature = transcript glyph = segments bgcolor = wheat fgcolor = black height = 10 key = Transcripts If you do not have a main subfeature, leave off the ``/mainmethod''. For example:
aggregator = transcript{TSS,exon,polyA}
A few formatting notes. You are free to mix simple and complex aggregators in the ``aggregator'' option. For example, you can activate the standard ``clone'' and ``alignment'' aggregators as well as the new transcript aggregator with a line like this one:
aggregator = clone
transcript{TSS,exon,polyA/primary_transc}
alignment
If the complex aggregator contains whitespace or apostrophes, you must surround it with double-quotes, like this:
"transcript{TSS,5'UTR,3'UTR,exon,polyA/primary_transc}"
Be aware that some glyphs look for particular method names when rendering aggregated features. For example, the standard ``transcript'' glyph is closely tied to the ``transcript'' aggregator, and looks for submethods named ``intron'', ``exon'' and ``CDS'', and a main method named ``transcript.'' Here is the list of available predefined aggregators:
alignment
clone
coding
transcript
none
orf
waba_alignment
wormbase_gene
To view the documentation for any of these aggregators, run the command ``perldoc Bio::DB::GFF::Aggregator::aggregator_name'', where ``aggregator_name'' is the name of the aggregator.
B9. GROUPING FEATURESgbrowse recognizes the concept of a ``group'' of related features that are connected by dotted lines. The canonical example is a pair of ESTs that are related by being from the two ends of the same cDNA clone. However many feature databases, including the GFF database recommended for gbrowse, do not allow for arbitrary hierarchical grouping. To work around this, you may specify a feature name-based regular expression that will be used to trigger grouping. It works like this. Say you are working with EST feature pairs and they follow the nomenclature 501283.5 and 501283.3, where the suffix is ``5'' or ``3'' depending on whether the read was from the 5' or 3' ends of the insert. To group these pairs by a dotted line, specify the ``group_pattern'' option in the appropriate track section:
group_pattern = /\.[53]$/
At render time, gbrowse will strip off this pattern from the names of all features in the EST track and group those that have a common base name. Hence 501283.5 and 501283.3 will be grouped together by a dotted line, because after the pattern is removed, they will share the same common name ``501283''. This works for all embedded pattern, provided that stripping out the pattern results in related features sharing the same name. For example, if the convention were ``est.for.501283'' and ``est.rev.501283'', then this grouping pattern would have the desired effect:
group_pattern = /\.(for|rev)\./
Don't forget to escape regular expression meta-characters and to consider the various ways in which the regular expression might break. It is entirely possible to create an invalid regular expression, in which case gbrowse will crash until you comment out the offending option.
B10. Linking-out from AUTO-generated detailsIf the [Track Defaults]-Link (see section B2) or an individual [Track]- Link (see section B3) is set to AUTO, the gbrowse_details script will be invoked when the user clicks on one of those features. This will generate a simple table of all feature information available in the database. This includes the user-defined tag/value attributes set in Column 9 of the GFF for that feature. It is possible to configure HTML link-outs for each tag-type in these tag/value pairs such that clicking on the value will invoke an external script with the tag and value information being passed to that script. This is achieved by adding a new stanza for each tag-type you wish to associate links to. The format of the stanza header is [TagName:DETAILS], and currently the only option allowed within this stanza is URL. For example, to link to a local cgi script from the following GFF line: IV curated exon 518 550 . + . Transcript B0273.1; local_id 11723 one might add the following stanza to the configuration file:
[local_id:DETAILS]
URL = http://localhost/cgi-bin/localLookup.cgi?tag=$tag;id=$value
The URL option's value should be a URL containing one or more variables. Variables begin with a dollar sign ($), and are replaced at run time with the information relating to the selected feature attribute. Recognized variables are:
$tag The "tag" of the tag/value pair
$value The "value" of the tag/value pair
C. GENERATING HISTOGRAMSWith a little bit of additional effort, you can set one or more tracks up to display a density histogram of the features contained within the track. For example, the human data source in GBrowse demo (http://www.wormbase.org/db/seq/gbrowse/human) uses density histograms in the chromosomal overview. In addition, when the features in the SNP track become too dense to view, this track converts into a histogram. To see this in action, turn on the SNP track and then zoom out beyond 150K. There are four steps for making histograms:
The first step is to generate the density data. Currently this is done by generating a GFF file containing a set of ``bin'' feature types. Use the generate_histogram.pl script to do this. You will find it in bioperl under the scripts/Bio-DB-GFF directory. Assuming that your database is named ``dicty'', you have a feature named SNP, and you wish to generate a density distribution across 10,000 bp bins, here is the command you would use: generate_histogram.pl -merge -d dicty -bin 10000 SNP >snp_density.gff This is saying to use the ``dicty'' database (-d) option, to use 10,000 bp bins (the -bin option) and to count the occurrences of the SNP feature throughout the database. In addition, the -merge option says to merge all types of SNPs into a single bin. Otherwise they will be stratified by their source. The resulting GFF file contains a series of entries like these ones: Chr1 SNP bin 1 10000 49 + . bin Chr1:SNP Chr1 SNP bin 10001 20000 29 + . bin Chr1:SNP What this is saying is that there are now a series of pseudo-features of type ``bin:SNP'' that occupy successive 10,000 bp regions of the genome. The score field contains the number of times a SNP was seen in that bin. You'll now load this file using load_gff.pl or fast_load_gff.pl: load_gff.pl -d dicty snp_density.gff The next step is to tell GBrowse how to use this information. You do this by creating a new aggregator for the SNP density information. Open the GBrowse configuration file and find the aggregators option. Add a new aggregator that looks like this:
aggregators = snp_density{bin:SNP}
This is declaring a new feature named ``snp_density'' that is composed of subparts of type bin:SNP. The last step is to declare a track for the density information. You will use the ``xyplot'' glyph, which can draw a number of graphs, including histograms. To add the SNP density information as a static track in the overview, create a section like this one: [SNP:overview] feature = snp_density glyph = xyplot graph_type = boxes scale = right bgcolor = red fgcolor = red height = 20 key = SNP Density This is declaring a new constant track in the overview named ``SNP Density.'' The feature is ``snp_density'', corresponding to the aggregator declared earlier. The glyph is ``xyplot'' using the graph type of ``boxes'' to generate a column graph. To set up a track so that the histogram appears when the user zooms out beyond 100,000 bp but shows the detailed information at higher magnifications, generate two track sections like these: [SNPs] feature = snp glyph = triangle point = 1 orient = N height = 6 bgcolor = blue fgcolor = blue key = SNPs [SNPs:100000] feature = snp_density glyph = xyplot graph_type = boxes scale = right The first track section sets up the defaults for the SNP track. SNPs are represented as blue triangles pointing North. The second track declaration declares that when the user zooms out to over 100K base pairs, GBrowse should display the snp_density feature using the xyplot glyph.
D. INTERNATIONALIZATIONGBrowse is partially internationalized. End-users whose browsers are set to request a non-English language will see the GBrowse main and secondary screens in their preferred language, provided that GBrowse has the appropriate translation file. Translation files are located in gbrowse.conf/languages/ and use the standard two-letter language abbreviations, such as ``fr'' for French, as well as the regional abbregiations, such as fr-CA for Canadian French. Currently there are translation files for French, Italian, and Japanese. If your favorite language isn't supported, you are encouraged to create a new translation file and contribute it to the GBrowse development effort. Please contact Lincoln Stein (lstein@cshl.org) for help in doing this. If the end user does not specify a preferred language, GBrowse will default to ``en'' (English). You can change this by placing a ``language'' option in the configuration file somewhere inside the [GENERAL] section. For example, to make Japanese the default, create this entry: language = ja GBrowse will still use the end-user's preferred language in preference to the default if the preferred language is available. Although GBrowse automatically changes the text and button language, it can't automatically translate the track labels. If you would like the track labels to localize, you will have to provide your own translations in the ``key'', ``citation'' and ``category'' options. The syntax is similar to that used for semantic zooming: [gene] glyph = transcript feature = transcript:curated height = 10 key = Named Gene key:fr = Gènes Nommés key:it = I Geni dati un nome a key:sp = Los Genes denominados category = Genes category:fr = Gènes The option is followed by a colon and the two-letter language name to indicate that when the page is being displayed with this language, to use the indicated value of the option. The option without the colon is the default. You may enter accented and umlauted characters directly, as shown, or use the HTML entities. Non-English character sets, such as Japanese, should also work correctly, provided that the translation file indicates the correct character set to use. HELP FILES: The GBrowse help files are in English. Although there is support for internationalizing the hep files, no one has done this yet. If you are industrious and wish to translate the help files into your favorite language, find the two help files where they are located in htdocs/gbrowse/. One is named general_help.html, while the other is named annotation_help.html. Translate them, and create new files with the language prefix appended to the end. For example, the French translation of annotation_help.html would be annotation_help.html.fr. LIMITATIONS: - There is no localization support. For example, GBrowse will print large numbers using commas (e.g. 1,234,567) instead of periods, even when talking to a European browser. - Although the HTML frame around the GBrowse genome image will use the appropriate character set, the overview and detail images themselves are limited to Latin alphabets. This is because of limited native character support in the GD library used by GBrowse. When a non-Latin character set is called for, such as Japanese, GBrowse will use Japanese for the frame, but English for the image. - The rate at which the GBrowse team adds new features to the browser often outstrips the ability of volunteers to update the translation files. This means that new buttons and fields may be displayed in English on an otherwise correctly internationalized page.
E. AUTHENTICATION AND AUTHORIZATIONYou can restrict who has access to gbrowse by IP address, host name, domain or username and password. Restriction can apply to the database as a whole, or to particular annotation tracks. To limit access to a whole database, you can use Apache's standard authentication and authorization. Gbrowse uses a URL of this form to select which database it is set to:
http://your.host/cgi-bin/gbrowse/your_database
where ``your_database'' is the name of the currently selected database. For example, the yeast database is http://your.host/cgi-bin/gbrowse/yeast. To control access to the entire database, create a <Location> section in httpd.conf. The <Location> section should look like this:
<Location /cgi-bin/gbrowse/your_database>
Order deny,allow
deny from all
allow from localhost .cshl.edu .ebi.ac.uk
</Location>
This denies access to everybody except for ``localhost'' and browsers from the domains .cshl.edu and .ebi.ac.uk. You can also limit by IP address, by username and password or by combinations of these techniques. See http://httpd.apache.org/docs/howto/auth.html for the full details. You can also limit individual tracks to certain individuals or organizations. Unless the stated requirements are met, the track will not appear on the main screen or any of the configuration screens. To set this up, add a ``restrict'' option to the track you wish to make off-limits:
[PROPRIETARY]
feature = etc
glyph = etc
restrict = Order deny,allow
deny from all
allow from localhost .cshl.edu .ebi.ac.uk
The value of the restrict option is identical to the Apache authorization directives and can include any of the directives ``Order,'' ``Satisfy,'' ``deny from,'' ``allow from,'' ``require valid-user'' or ``require user.'' The only difference is that the ``require group'' directive is not supported, since the location of Apache's group file is not passed to CGI scripts. Note that username/password authentication must be turned on in httpd.conf and the user must have successfully authenticated himself in order for the username to be available. As with other gbrowse options, restrict can be a code subroutine. The subroutine will be called with three arguments consisting of the host, ip address and authenticated user. It should return a true value to allow access to the track, or a false value to forbid it. This can be used to implement group-based authorization or more complex schemes. Here is an example that uses the Text::GenderFromName to allow access if the user's name sounds female and forbids access if the name sounds male. (It might be useful for an X-chromosome annotation site.)
restrict = sub {
my ($host,$ip,$user) = @_;
return unless defined $user;
use Text::GenderFromName qw(gender);
return gender($user) eq 'f';
}
You should be aware that the username will only be defined if username authentication is turned on and the user has successfully authenticated himself against Apache's user database using the correct password. In addition, the hostname will only be defined if HostnameLookups have been turned on in httpd.conf. In the latter case, you can convert the IP address into a hostname using this piece of code:
use Socket;
$host = gethostbyaddr(inet_aton($addr),AF_INET);
Note that this may slow down the response time of gbrowse noticeably if you have a slow DNS name server. Another thing to be aware of when restricting access to an entire database is that that even though the database itself will not be accessible to unauthorized users, the name of the database will still be available from the popup ``Data Source'' menu. If you wish even the name to be suppressed from view by unauthorized users, add the following line to the [GENERAL] section of the configuration file of the database you wish to suppress:
restrict = require valid-user
The syntax described earlier for restricting access to tracks by hostname, IP address or username holds true for restricting the visibility of the database on the Data Source popup menu.
F. DISPLAYING GENETIC AND RH MAPSGBrowse can be tweaked to make it more suitable for displaying genetic and radiation hybrid maps. The main issue is that the Bio::DB::GFF database expects coordinates to be positive integers, not fractions, but genetic and RH maps use floating point numbers. Working around this is a bit of an ugly hack. Before loading your data you must multiply all your coordinates by a constant power of 10 in order to convert them into integers. For example, if a genetic map uses Morgan units ranging from 0 to 1.80, you would multiple by 100 to create a map in ranging from 0 to 180. Create a GFF file containing the markers in modified coordinates and load it as usual. Now you must tell GBrowse to reverse these changes. Enter the following options into the [GENERAL] section of the configuration file: units = M unit_divider = 100 These two options tell GBrowse to use ``M'' (Morgan) units, and to divide all coordinates by 100. GBrowse will automatically display the scale using the most appropriate units, so the displayed map will typically be drawn using cM units.
G. CHANGING THE LOCATION OF THE CONFIGURATION FILESIf you wish to change the location of the gbrowse.conf configuration file directory, you must manually edit the gbrowse CGI script. Open the script in a text editor, and find this section: ################################################################### # Non-modperl users should change this variable if needed to point # to the directory in which the configuration files are stored. # use constant CONF_DIR => '/usr/local/apache/conf/gbrowse.conf'; # ################################################################### Change the definition of CONF_DIR to the desired location of the configuration files. An alternative, for users of mod_perl only, is to add the GBrowseConf per-directory variable to the configuration for the directory in which the gbrowse script lives. This variable overrides the CONF_DIR value. For example: <Directory /usr/local/apache/cgi-perl> SetHandler perl-script PerlHandler Apache::Registry PerlSendHeader On Options +ExecCGI PerlSetVar GBrowseConf /etc/gbrowse.conf </Directory>
H. USING DAS (DISTRIBUTED ANNOTATION SYSTEM) DATABASESGBrowse has limited DAS support, allowing it to visualize data from a single DAS source. To get this support, you must use Bio::Das version 0.90 or higher, available from http://www.biodas.org (look for the ``Bio-Das2'' library). A sample [GENERAL] configuration section looks like this: [GENERAL] description = Das Example Database (dicty) db_adaptor = Bio::Das db_args = -source http://www.biodas.org/cgi-bin/das -dsn dicty The db_adaptor option must be set to ``Bio::Das''. The db_args option must contain a -source pointing to the base of the remote DAS server, and a -dsn pointing to the name of the annotation database. The remainder of the configuration file should be configured as described earlier. The following short script will return a list of the feature types known to the remote DAS server. You can use the output of this script as the basis for the tracks to configure. #!/usr/bin/perl use strict;
use Bio::Das;
my $db = Bio::Das->new('http://localhost/cgi-bin/das'=>'dicty');
print join "\n",$db->types;
Limitations: The DAS implementation does not descend into subcomponents. For example, if the user requests features on a chromosome, but the remote DAS server has annotated genes using contig coordinates, then the genes will not appear on the chromosome. The gbrowse_details script does not provide useful information because the DAS/1 protocol does not provide a way to retrieve attribute information on a named feature.
I. THE BioMOBY BROWSERThe BioMOBY project aims to design and deploy platforms that enable and simplify biological database interoperability. To date, the MOBY-Services (MOBY-S) branch of the BioMOBY project has published a fairly stable API that is now being used by data providers worldwide to publish their data in an interoperable manner. A simple MOBY browser has been written for Gbrowse that allows the end-user to ``surf'' out of their Gbrowse view and begin exploring data related to the genomic features displayed in Gbrowse. Configuration of the gbrowse_moby script does, at this time, require some VERY simple code-editing, and small modifications to your XX.organism.conf configuration file. These are described in detail below:
J. BioMOBY ServicesA selection of services are distributed with the Gbrowse package that will allow you to serve your underlying data using the BioMOBY Services architecture. To enable these, simply do the following:
K. FILTERING SEARCH RESULTSGBrowse provides a method to filter the contents of individual tracks based on information that can be obtained from feature attributes. For example, suppose you have performed a blast and added all hits as similarity features on an entry. In gbrowse, all those features can get a little crowdy. The administrator can decide to show only the top 5 of the blast hits. This can easily be accomplished by adding the filter option in the conf file. It might look like this:
[BLAST]
feature = blast
glyph = segments
filter = sub {
my $feat = shift;
(my $rank) = $feat->get_tag_values('rank'); # persistent Bio::SeqFeature::Generic features
#(my $rank) = $feat->attributes('rank'); # Bio::DB::GFF::Feature
$rank < 6;
}
Another useful example is to show features coming from a plain genbank file. When loaded into BioSQL the source becomes 'EMBL/Genbank/SwissProt'. Using the Bio::DB::Das::BioSQL adaptor you have to pass the source to the feature option. It can be rather difficult to distinguish all the features when they all have the same source string. This problem can be solved using the filter option. In the following example the difference between the features is done based on the primary_tag
[REGION]
feature = EMBL/GenBank/SwissProt
filter = sub {
my $feat = shift;
$feat->primary_tag =~ /region/i;
}
key = RefSeq Protein Domains
[SIGPEPTIDE]
feature = EMBL/GenBank/SwissProt
filter = sub {
my $feat = shift;
$feat->primary_tag =~ /sig_peptide/i;
}
key = RefSeq Signal Peptide
L. INVOKING GBROWSE URLs (under construction)This section describes the public CGI parameters recognized by GBrowse. By setting the parameters in the URL, you can get gbrowse to do various useful things:
M. FURTHER INFORMATIONFor further information, bug reports, etc, please consult the mailing lists at www.gmod.org. The main mailing list for gbrowse support is gmod-gbrowse@lists.sourceforge.net. Have fun! Lincoln Stein & the GMOD development team lstein@cshl.edu |
|
|
|
cain@cshl.org |