Genome-preview machine

In early 2011, we created a new machine called genome-preview, which is a mirror of our development machine: genome-test.

It is here: http://genome-preview.soe.ucsc.edu/

It has these warnings on the home page:

WARNING: This is our preview site. This website is a weekly mirror of our internal development server for public access. Data and tools here are under construction, have not been quality reviewed, and are subject to change at any time. We provide this site for early access, with the warning that it is less available and stable than our public site. For high-quality reviewed annotations on our production server, visit http://genome.ucsc.edu.

In general, we do not push updates/files/tables/patches to this machine by hand. It is generally understood that it will sometimes be out of commission, as it is a mirror of a development machine.

The impetus behind this machine was to allow early access to the ENCODE data by ENCODE consortia members.

Automated Updates

The genome-preview machine is updated on a weekly basis like so:

On Friday nights (11:33 pm), the following stuff is copied from hgwdev:

• mysql tables (excluding mysql, customTrash, and other things)
• hgcentraltest tables (excluding userDb, sessionDb, namedSessionDb and hubStatus)
• CGIs
• htdocs

Most of the stuff on /gbdb comes from backups of the /hive/data and /hive/groups.

• /hive/data is backed up on Fridays at 5:22 am
• /hive/groups is backed up on Wednesdays at 5:22 am.

Note that this may cause some inconsistencies if new data is added on Thursday to /hive/groups and accessed by new CGIs, since the data will not be available on the backups until the following Wednesday.

Updating Genome-Preview with pushDevToPreview

If there is a need to sync the CGIs on hgwdev to genome-preview the pushDevToPreview script will do it (from the qateam@hgwdev identity).

 pushDevToPreview -h
Usage: pushDevToPreview [-chiqvy] 

  -c | --check    : Don't do the push, just say what would be pushed.
  -h | --help     : Print this message.
  -i              : Interactive mode. Ask me before doing any command.
  -q | --quiet    : Quiet mode. Don't print anything unless necessary.
  -v | --verbose  : Increase verbosity.
  -y | --noprompt : Just do it, don't ask me.

Note: you'll first need to update the js files for preview at: ls -l /usr/local/apache/htdocs/js/preview/

git pull
cd ~/kent/src/hg/js
make USER=preview

Note: you'll first need to update the style files for hgwdev at: ls -l /src/hg/htdocs/style

git pull
cd ~/kent/src/hg/htdocs/style
make alpha

Note: you'll first need to update the CGIs for preview at: ls -lrt /usr/local/apache/cgi-bin-preview/

git pull
$ cd ~/kent/src
$ USER=preview make cgi

There are some redundant git pulls here, they exist to dry home the message to be sure you are up to date. Once all your files are updated, you should run the command with the -c --check option to see what will be updated: pushDevToPreview -c

If things look pretty good, then you are ready to run the command, with the -i interactive mode so you can see what commands are being run:

 pushDevToPreview -i

Pushing files from hgwdev to genome-preview

Do you wish to proceed? (y/[n]): y

Doing directory /usr/local/apache/cgi-bin-preview/ between hgwdev and genome-preview


The command to be executed is /usr/bin/rsync -rlptDi --delete --chown=qateam:protein --exclude=hg.conf*  /usr/local/apache/cgi-bin-preview/ genome-preview:/usr/local/apache/cgi-bin-preview/
...
...