Selenium python GBiB script

From Genecats
Jump to navigationJump to search

Overview

See the Selenium page for more info about Selenium, but note it will be outdated.

This page has an older example script that is now obsolete and a newer script.

The older script runs on python2.7 with installed selenium (pip install selenium) .

Below is also a section about adapting this script to point it to the RR.

Then after the older scripts, there is a newer superior script genomeTest.py that was working as of May 2017.

OLDER section

You can likely ignore all this information now.

OLDER Running Script

python gbib.selenium.py

Results desired: 

$python gbib.selenium.py
.
----------------------------------------------------------------------
Ran 1 test in 106.073s

OK
  • Be sure to watch the script run, it isn't verifying results happen.

OLDER Script Code

This code probably has imperfections, it is copy of an earlier script pointed to genome-euro with the http replaced with GBiBs 127.0.0.01:1234 address.

The process to build it was to use FireFoxes Selenium plug-in to record steps. Then to use the plug-in to export it as python. I recall it needed some changes to work with python2.7. I tested that I could cut and paste the below and execute the code in my environment (Python 2.7.6 with the pip install of selenium). 

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
import unittest, time, re

class Genome(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(30)
        self.base_url = "http://127.0.0.1:1234/cgi-bin/hgGateway"
        self.verificationErrors = []
        self.accept_next_alert = True
    
    def test_genome(self):
        driver = self.driver
        driver.get(self.base_url + "Be sure you are not being redirected to genome vs. genome-euro")
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway")
        # ERROR: Caught exception [ERROR: Unsupported command [selectWindow | null | ]]
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway")
        driver.find_element_by_id("home-link").click()
        driver.find_element_by_link_text("Genome Browser").click()
        driver.find_element_by_link_text("Genomes").click()
        driver.find_element_by_link_text("Click here to reset").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway?db=hg19")
        Select(driver.find_element_by_name("org")).select_by_visible_text("Mouse")
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("Genomes").click()
        Select(driver.find_element_by_css_selector("select[name=\"db\"]")).select_by_visible_text("July 2007 (NCBI37/mm9)")
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("Genomes").click()
        driver.find_element_by_link_text("Click here to reset").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway?db=hg19")
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_xpath("//td[@id='td_data_knownGene']/div[2]/map/area[4]").click()
        driver.find_element_by_link_text("Genome Browser").click()
        driver.find_element_by_css_selector("map[name=\"map_data_refGene\"] > area.area").click()
        driver.find_element_by_css_selector("map[name=\"map_data_refGene\"] > area.area").click()
        driver.find_element_by_link_text("Genome Browser").click()
        driver.find_element_by_css_selector("input[type=\"button\"]").click()
        driver.find_element_by_name("hgct_customText").clear()
        driver.find_element_by_name("hgct_customText").send_keys("http://hgwdev.gi.ucsc.edu/~brianlee/customTracks/examples.txt")
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_name("submit").click()
        driver.find_element_by_link_text("Genomes").click()
        driver.find_element_by_link_text("Click here to reset").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway?db=hg19")
        driver.find_element_by_xpath("//input[@value='track hubs']").click()
        driver.find_element_by_link_text("My Hubs").click()
        driver.find_element_by_id("hubUrl").clear()
        driver.find_element_by_id("hubUrl").send_keys("http://hgwdev.gi.ucsc.edu/~brianlee/examples/hubExample/hub.txt")
        driver.find_element_by_name("hubAddButton").click()
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("Genomes").click()
        driver.find_element_by_link_text("Click here to reset").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway?db=hg19")
        driver.get("http://127.0.0.1:1234/cgi-bin/hgBlat")
        driver.find_element_by_name("userSeq").clear()
        driver.find_element_by_name("userSeq").send_keys("AACAAAATCAAACTGTTTTTGTTGGACAATTCTCTGTTAAGCAGCTATAA\nGCTGAATGACATTAACCGCAAAATGTAACCATAAAGGCCATAAACCCGAC\nATTGTTAATTAATTAAATGCCTCATTAACTTTTTTAAAAACATGATTTAT\nTCGATTCATAGAAAACTTAACCATCACTACTAAATGCACACACATGCGGT\nTCCACATTGGCATCTTAGCCTAAGAACAGACAGGTTCAACTGTAACTGGC\nCTTTCAGGTGGTCTATTACAGATCTGAAGACAGAGGGTGTTTCTAAACCT\nCAAGAACCAGATTAACAGAAAACAAAGCTTGAGCAGCCTTTTTATTGCAT\nGTGGTATCTTTTTAGCTAAGCAGAAGACAATGATAAAGAGGGGTTTTGGG\nAAACCTCTCCCAAAGCTGTGCATTCATACCGTACCTTATCCTGTTAAGCA\nAACTGTTCTTTTATTTTAAAGGGTTTACACTGCCACATCTGAATGGACTA")
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("browser").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgPcr")
        driver.find_element_by_name("wp_f").clear()
        driver.find_element_by_name("wp_f").send_keys("AACAAAATCAAACTGTTTTTGTTGGACAATTCTCTGTTAAGCAGCTATAA")
        driver.find_element_by_name("wp_r").clear()
        driver.find_element_by_name("wp_r").send_keys("AACTGTTCTTTTATTTTAAAGGGTTTACACTGCCACATCTGAATGGACTA")
        driver.find_element_by_name("wp_flipReverse").click()
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("chrX:39918932+39919431").click()
        driver.find_element_by_link_text("Genomes").click()
        driver.find_element_by_link_text("Click here to reset").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgGateway?db=hg19")
        driver.find_element_by_name("Submit").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgConvert")
        Select(driver.find_element_by_name("hglft_toOrg")).select_by_visible_text("Human")
        Select(driver.find_element_by_name("hglft_toOrg")).select_by_visible_text("Mouse")
        Select(driver.find_element_by_name("hglft_toDb")).select_by_visible_text("July 2007 (NCBI37/mm9)")
        driver.find_element_by_name("hglft_doConvert").click()
        driver.find_element_by_link_text("chr16:90220619-90227077").click()
        driver.find_element_by_link_text("Genome Browser").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgMirror")
        driver.find_element_by_link_text("Predefined tracks sets").click()
        driver.find_element_by_link_text("Genome Browser").click()
        driver.get("http://127.0.0.1:1234/cgi-bin/hgLiftOver")
        Select(driver.find_element_by_name("hglft_fromOrg")).select_by_visible_text("Human")
        driver.find_element_by_name("hglft_userData").clear()
        driver.find_element_by_name("hglft_userData").send_keys("chr21:33,031,597-33,041,570")
        driver.find_element_by_name("hglft_multiple").click()
        driver.find_element_by_name("Submit").click()
        driver.find_element_by_link_text("View Conversions").click()
        driver.find_element_by_link_text("Genome Browser").click()
        self.assertEqual("chr16:90,220,619-90,227,077", driver.find_element_by_id("positionDisplay").text)
    
    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException as e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException as e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()

OLDER: Adapting Script to Point to RR

You can take this script and copy it to point to the RR.

  1. Create a directory that you put your RR directed scripts into, I called mine hgw0.6
  2. Create a text file of the above script copied and pasted
    1. If you try to run this file, you must have your GBiB up and running as the URLs point to http://127.0.0.1:1234/cgi-bin/hgGateway
    2. In the next step you will swap these URLs to point to hgw0-6
    3. Before you do that remove the lines that point to the hgMirror GBiB-only CGI
      1. driver.get("http://127.0.0.1:1234/cgi-bin/hgMirror")
      2. driver.find_element_by_link_text("Predefined tracks sets").click()
      3. Copy your test file and name it hgw0.selenium.py or run this loop ( for i in {0..6}; do cp test.py hgw$i.selenium.py; done )
  3. Now run this perl statement to swap the GBIB address for the hgw0 address
    1. perl -p -i -e 's#127.0.0.1:1234#hgw0.cse.ucsc.edu#' hgw0.selenium.py
      1. OR run this loop for i in {0..6}; do echo "perl -p -i -e 's#127.0.0.1:1234#hgw$i.cse.ucsc.edu#' hgw$i.selenium.py" >> script; done
      2. chmod 700 script;
      3. run ./script
  4. Repeat the above steps of copying your test file, and swapping the address so you create scripts for all machines (if you didn't do the loop)
    1. For example, create hgw1.selenium.py
    2. Then run perl -p -i -e 's#127.0.0.1:1234#hgw1.cse.ucsc.edu#' hgw1.selenium.py
    3. At the end you will have seven selenium scripts for hgw0.... to hgw6.selenium.py
  5. Lastly you can run them back to back with this command. Check for the "OK" meaning it completed fine.
    1. for i in {0..6}; do echo;echo;echo;echo Running hgw$i test; python hgw$i.selenium.py; done
      1. NOTE: I put this is a script, and then I need to specifically call out the python version I am running
      2. for i in {0..6}; do echo;echo;echo;echo Running hgw$i test; python3.4 hgw$i.selenium.py; done

genomeTest.py

You need to install python3 for this to work and then pip3 install selenium and then download and install chrome and a chrome driver. 


#This requires you to have /Users/brianlee/Downloads/chromedriver installed.
#You can get this from https://sites.google.com/a/chromium.org/chromedriver/downloads
# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re, sys

class GenomeTest(unittest.TestCase):
    def __init__(self, testname, host):
        super(GenomeTest, self).__init__(testname)
        self.base_url = host
        self.driver = webdriver.Chrome("/Users/brianlee/Downloads/chromedriver")
        self.driver.implicitly_wait(30)
        self.verificationErrors = []
        self.accept_next_alert = True
    
    def test_genome_test(self):
        #print("url is %s\n" % self.host)
        driver = self.driver
        driver.get(self.base_url + "/index.html")
        driver.find_element_by_link_text("Home").click()
        driver.find_element_by_link_text("Genomes").click()
        if self.base_url[-20:] == "genome-euro.ucsc.edu": # genome-euro
            driver.find_element_by_link_text("genome-euro.ucsc.edu").click()
        if self.base_url[-20:] == "genome-asia.ucsc.edu": # genome-asia
            driver.find_element_by_link_text("genome-asia.ucsc.edu").click()
        self.cart_reset(driver)
        Select(driver.find_element_by_id("selectAssembly")).select_by_visible_text("Feb. 2009 (GRCh37/hg19)")
        driver.find_element_by_css_selector("div.jwGoButton").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_xpath("//td[@id='td_data_knownGene']/div[2]/map/area[5]").click()
        driver.get(self.base_url + "/cgi-bin/hgGateway?db=mm10")
       
        # check mm10 hgTracks and hgGene 
        self.driver.implicitly_wait(1000)
        self.hover_over_menu(driver, "//li[@id='tools1']", "//li[@id='tools1']/ul/li[3]") # click mm10 from Genomes drop down
        driver.find_element_by_xpath("//td[@id='td_data_knownGene']/div[2]/map/area[6]").click() # clicks into hgGene
        
        # test custom track
        driver.get(self.base_url + "/cgi-bin/hgGateway?db=hg19")
        self.driver.implicitly_wait(1000)
        if self.base_url[-4:] == "1234": # gbib
            self.hover_over_menu(driver, "//li[@id='myData']", "//li[@id='myData']/ul/li[5]")
        else: # any other mirror
            self.hover_over_menu(driver, "//li[@id='myData']", "//li[@id='myData']/ul/li[4]")
        
        driver.find_element_by_name("hgct_customText").clear()
        driver.find_element_by_name("hgct_customText").send_keys("http://hgwdev.gi.ucsc.edu/~brianlee/customTracks/examples.WITHOUT.FTPS.txt")
        driver.find_element_by_name("Submit").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_name("submit").click()

        self.cart_reset(driver)  #Test mm10 ENCODE hub
        driver.get(self.base_url + "/cgi-bin/hgTracks?db=mm10&hideTracks=1")
        driver.get(self.base_url + "/cgi-bin/hgGateway?db=mm10&hubUrl=https://www.encodeproject.org/experiments/ENCSR736GVO/@@hub/hub.txt")
        driver.get(self.base_url + "/cgi-bin/hgHubConnect?#unlistedHubs")
        driver.find_element_by_link_text("My Hubs").click()
        self.assertEqual("ENCODE Data Coordination Center Data Hub", driver.find_element_by_xpath("//table[@id='unlistedHubsTable']/tbody/tr/td[3]").text)
        driver.get(self.base_url + "/cgi-bin/hgTracks")
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("chr12:56,694,976-56,714,605")
        driver.find_element_by_id("hgt.jump").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_id("hgt.out1").click()

        self.cart_reset(driver)  #Test AssemblyHub search 
        driver.get(self.base_url + "/cgi-bin/hgGateway?hubUrl=http://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/vertebrate_mammalian/hub.ncbi.txt&genome=GCA_000493695.1_BalAcu1.0&position=lastDbPos")
        driver.get(self.base_url + "/cgi-bin/hgTracks")
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("ATDI01079686")
        driver.find_element_by_id("hgt.jump").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_id("hgt.out1").click()

        driver.get(self.base_url + "/cgi-bin/cartReset")  # Test HGVS searches 
        driver.get(self.base_url + "/cgi-bin/hgTracks?db=hg38")
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_000310.3(PPT1):c.271_287del17insTT")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_007262.4(PARK7):c.-24+75_-24+92dup")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_006172.3(NPPA):c.456_*1delAA")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("MYH11:c.503-14_503-12del")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_198576.3(AGRN):c.1057C>T")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_198056.2:c.1654G>T")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NP_002993.1:p.Asp92Glu")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NP_002993.1:p.D92E")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("BRCA1 Ala744Cys")
        driver.find_element_by_id("hgt.jump").click()
        driver.find_element_by_id("positionInput").clear()
        driver.find_element_by_id("positionInput").send_keys("NM_000828.4:c.-2G>A")
        driver.find_element_by_id("hgt.jump").click()
  
        self.cart_reset(driver)
        driver.get(self.base_url + "/cgi-bin/hgGateway?db=hg19")
        self.hover_over_menu(driver, "//li[@id='tools3']", "//li[@id='tools3']/ul/li[1]") # hgBlat
        driver.find_element_by_name("userSeq").clear()
        driver.find_element_by_name("userSeq").send_keys("AACAAAATCAAACTGTTTTTGTTGGACAATTCTCTGTTAAGCAGCTATAA\\nGCTGAATGACATTAACCGCAAAATGTAACCATAAAGGCCATAAACCCGAC\\nATTGTTAATTAATTAAATGCCTCATTAACTTTTTTAAAAACATGATTTAT\\nTCGATTCATAGAAAACTTAACCATCACTACTAAATGCACACACATGCGGT\\nTCCACATTGGCATCTTAGCCTAAGAACAGACAGGTTCAACTGTAACTGGC\\nCTTTCAGGTGGTCTATTACAGATCTGAAGACAGAGGGTGTTTCTAAACCT\\nCAAGAACCAGATTAACAGAAAACAAAGCTTGAGCAGCCTTTTTATTGCAT\\nGTGGTATCTTTTTAGCTAAGCAGAAGACAATGATAAAGAGGGGTTTTGGG\\nAAACCTCTCCCAAAGCTGTGCATTCATACCGTACCTTATCCTGTTAAGCA\\nAACTGTTCTTTTATTTTAAAGGGTTTACACTGCCACATCTGAATGGACTA")
        driver.find_element_by_name("Submit").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_link_text("browser").click()
        driver.find_element_by_id("p_btn_hgUserPsl").click()
        self.driver.implicitly_wait(1000)
        self.cart_reset(driver)
        if self.base_url[-4:] == "1234":
            self.hover_over_menu(driver, "//li[@id='tools3']", "//li[@id='tools3']/ul/li[6]") # hgPcr
        else:
            self.hover_over_menu(driver, "//li[@id='tools3']", "//li[@id='tools3']/ul/li[7]") # hgPcr
        driver.find_element_by_name("wp_f").clear()
        driver.find_element_by_name("wp_f").send_keys("AACAAAATCAAACTGTTTTTGTTGGACAATTCTCTGTTAAGCAGCTATAA")
        driver.find_element_by_name("wp_r").clear()
        driver.find_element_by_name("wp_r").send_keys("AACTGTTCTTTTATTTTAAAGGGTTTACACTGCCACATCTGAATGGACTA")
        driver.find_element_by_name("wp_flipReverse").click()
        driver.find_element_by_name("Submit").click()
        self.driver.implicitly_wait(1000)
        driver.find_element_by_link_text("chrX:40059679+40060178").click()
        self.driver.implicitly_wait(1000)
        self.cart_reset(driver)
        driver.find_element_by_link_text("Genome Browser").click()
        self.hover_over_menu(driver, "//li[@id='view']", "//li[@id='view']/ul/li[3]") # hgConvert
        driver.find_element_by_name("hglft_doConvert").click()
        driver.find_element_by_link_text("chr9:136127387-136156434").click()
        driver.find_element_by_css_selector("#tools3 > span").click()
        if self.base_url[-4:] == "1234":
            self.hover_over_menu(driver, "//li[@id='tools3']", "//li[@id='tools3']/ul/li[7]") # hgLiftOver
        else:
            self.hover_over_menu(driver, "//li[@id='tools3']", "//li[@id='tools3']/ul/li[8]") # hgLiftOver
        driver.find_element_by_name("hglft_userData").clear()
        driver.find_element_by_name("hglft_userData").send_keys("chr21:33,031,597-33,041,570")

    def hover_over_menu(self, driver, menu_header, menu_item_xpath):
        """hovers over menubar and clicks an item in a dropdown list"""
        # below code taken from: 
        # http://stackoverflow.com/questions/27934945/selenium-move-to-element-does-not-always-mouse-hover
        men_menu = WebDriverWait(driver,10).until(EC.visibility_of_element_located((
            By.XPATH, menu_header)))
        ActionChains(driver).move_to_element(men_menu).perform() 
        link_to_click = WebDriverWait(driver,10).until(EC.visibility_of_element_located(
            (By.XPATH, menu_item_xpath)))
        link_to_click.click()
    def cart_reset(self, driver):
        # hovers over the Genome Browser menubar item and clicks cart reset
        self.hover_over_menu(driver, "//li[@id='tools2']", "//li[@id='tools2']/ul/li[3]") 

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException as e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException as e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    suite = unittest.TestSuite()
    suite.addTest(GenomeTest("test_genome_test", sys.argv[1]))
    unittest.TextTestRunner().run(suite)

Running the program

Give the program any URL http:genome-euro/asia or hgw0/hgwbeta/hgwdev. 

python genomeTest.py http:hgwdev.gi.ucsc.edu
.
----------------------------------------------------------------------
Ran 1 test in 78.135s

OK
Retrieved from "https://genomewiki.ucsc.edu/genecats/index.php?title=Selenium_python_GBiB_script&oldid=4881"