How To Get Internet Speed Results From Fast.com

May 26, 2024 Post a Comment

I want to regularly check my internet speed and reset my router if it drops a certain threshold, which seems to fix the crappy connection my ISP 'provides'. Although there might b

Solution 1:

EDIT 2020-11-03: I've fixed the code, and it works again thanks to Carlos.

The page is JS-driven, and there's no easy way to get the results by simply scraping the HTML off the page, so requests and bs4 alone won't help you. You need to run a full modern browser to let the test run, and wait however long it needs and then get the results.

Setup

Ok, let's use Selenium for this, which can control Chrome, Firefox, Safari, basically any browser under the sun.

Install the package by running:

pip install selenium

We also need to install a driver to control our browser, you can find it here (In fact read the page about installation completely, it has everything you need to get it running). Then we need to place that executable in our PATH. Easiest way is to just put it under c:\Windows on Windows or /usr/bin on Linux. But there's tons of documentation on SO and in the internet, so learn the proper way to do this correctly, you'll need it.

With that out of the way, we also need Beautiful Soup, which is the defacto HTML parser (on that note, we can keep using selenium and browser, but I'm used to using bs4 for this job). Install it using

pip install bs4

Running the test

Now to get the results, we need to run the browser, go to https://fast.com, let the test finish, then get the HTML for the results, then extract the info we need.

How do we know when the test is finished?

Well, we could just wait, like 30s until everything finishes. But what if it finishes early? Or not finish at all? Then we'd have waited for no reason. There's a better way.

When the test finishes, the spinner turns green. And if we watch the DOM from the Developer Console, we see that it gets a succeeded class.

But if we expand the results, we see the upload results are not in yet, when that happens, the page is updated once more and we reach at this state:

Baca Juga

Waits

Selenium has an explicit wait feature that lets you wait until something happens on the page. We'll use it to check and wait until some element with .succeeded class is present on the page. If you just need the download speeds, simply wait for spinner to get .succeeded class, if you also need the uploads results, you need to wait for that instead. For this job we can use this helper function:

from selenium.common.exceptions import TimeoutException
from selenium.webdriver import Chrome
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait


defwait_until_present(driver: Chrome, selector: str, timeout: int = 5):
    condition = EC.presence_of_element_located((By.CSS_SELECTOR, selector))
    try:
        WebDriverWait(driver, timeout).until(condition)
    except TimeoutException as e:
        raise LookupError(f'{selector} is not present after {timeout}s') from e

Extracting the results

Once the results are in, we get the HTML for the parent element that contains both upload and download results.

# this is the parent element that contains both download and upload results
results_selector = '.speed-controls-container'
results_el = driver.find_element_by_css_selector(results_selector)
results_html = results_el.get_attribute('outerHTML')

Then we feed the HTML to BeautifulSoup and extract the values.

Code

Here's the code in its entirety:

from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
from contextlib import contextmanager


@contextmanagerdefget_chrome() -> Chrome:
    # https://docs.python.org/3.7/library/contextlib.html#contextlib.contextmanager
    opts = ChromeOptions()
    # opts.headless = True
    driver = Chrome(options=opts)
    yield driver
    driver.close()


defwait_until_present(driver: Chrome, selector: str, timeout: int = 5):
    condition = EC.presence_of_element_located((By.CSS_SELECTOR, selector))
    try:
        WebDriverWait(driver, timeout).until(condition)
    except TimeoutException as e:
        raise LookupError(f'{selector} is not present after {timeout}s') from e


defextract_speed_info(soup: BeautifulSoup) -> dict:
    dl_speed = soup.select_one('#speed-value').text
    dl_unit = soup.select_one('#speed-units').text
    upload_speed = soup.select_one('#upload-value').text
    upload_unit = soup.select_one('#upload-units').text

    return {
        'upload': f'{upload_speed}{upload_unit}',
        'download': f'{dl_speed}{dl_unit}'
    }


defrun_speed_test() -> dict:
    with get_chrome() as driver:
        driver.get('https://fast.com')

        # wait at most 60s until upload results come in
        download_done_selector = '#speed-value.succeeded'
        upload_done_selector = '#upload-value.succeeded'
        wait_until_present(driver, upload_done_selector, timeout=60)

        # this is the parent element that contains both download and upload results
        results_selector = '.speed-container'
        results_el = driver.find_element_by_css_selector(results_selector)
        results_html = results_el.get_attribute('outerHTML')

    # we're finished with chrome, let it close (by exiting with block)

    soup = BeautifulSoup(results_html, 'html.parser')
    info = extract_speed_info(soup)
    return info


if __name__ == '__main__':
    try:
        results = run_speed_test()
        print('Speed results:', results)
    except LookupError as e:
        print('Cannot get speed results')
        print(e)
        exit(1)

output:

{'upload': '320 Kbps', 'download': '3.4 Mbps'}

Solution 2:

There is a command line client for fast.com written in golang: https://github.com/ddo/fast/releases

Solution 3:

The answer provided by abdusco was very helpful. Anyway, now in 2020 it doesn't work. To load the upload result it is necessary to click on the "show more information" button after the download test is completed. To fix that, just add another wait_visble after loading the page and click the button.

This is the code that must be modified from abdusco's answer:

    upload_done_selector = '#upload-value.succeeded'
    more_info_selector = "show-more-details-link"
    wait_visible(driver, "#"+more_info_selector, timeout=60)
    driver.find_element_by_id(more_info_selector).click()
    wait_visible(driver, upload_done_selector, timeout=60)

lacucinadiadine