Please help contribute to the Reddit categorization project here

    lieutenant_lowercase

    + friends - friends
    15,472 link karma
    5,845 comment karma
    send message redditor for

    [–] Trying to scrape uniform data from multiple pages with a for loop using beautifulsoup lieutenant_lowercase 1 points ago in learnpython

    I thought it was this. I wrote some code at work which is about 6 lines that will get all the data for you, including extra info the site doesn't display. You don't need selenium, the data is loaded dynamically through an API call and returns JSON. I will send it you tomorrow when I'm back home. 99% of the time you don't need selenium, just browse the site with the chrome developer console open (F12) and it will show how the page is generating content. It makes retrieving data a million times easier.

    [–] Armed Police at Oxford Circus lieutenant_lowercase 3 points ago * (lasted edited 16 hours ago) in london

    Re: Olly Murs. Yes I agree he has a responsibility to not tweet bullshit but if he was in Selfridges and a bunch of people come running in screaming saying there has been gunshots outside, I don't really blame him for his tweet, he was only trying to help despite helping whip up the mass hysteria.

    [–] Armed Police at Oxford Circus lieutenant_lowercase 20 points ago in london

    I am in Old Street.

    lol how is that relevant

    [–] how to parse with selenium lieutenant_lowercase 1 points ago in learnpython

    You are giving me so little information to work on I can't help you.

    I'm assuming your code should be:

    url = 'http://scene-rls.net/category/movies/'
    browser.get(url)
    titles = browser.find_elements_by_class_name('postHeader')
    for title in titles:
        print(title.find_element_by_tag_name('a').text)
    

    [–] how to parse with selenium lieutenant_lowercase 1 points ago in learnpython

    Doesn't work how? What is in titles?

    [–] how to parse with selenium lieutenant_lowercase 2 points ago in learnpython

    <a> isn't css so you shouldn't be using

    find_element_by_css_selector
    

    use

    find_element_by_tag_name 
    

    instead

    [–] Selecting rows in Pandas given condition a or condition b lieutenant_lowercase 1 points ago in learnpython

    Also can do:

    dfs = df[(df["B"] > 30) | (df["C"] == 700)]["B","D","E"]
    

    Always more than one way to do something in Pandas!

    [–] How to get kps upload speed and mbs from a query string? lieutenant_lowercase 2 points ago in learnpython

    Correct but I assumed you could do that yourself?

    Total KBs transfered / Total seconds elapsed between calls

    [–] A Beginner Python Enthusiast! lieutenant_lowercase 1 points ago in learnpython

    Question been asked many times on this subreddit or the /r/datascience subreddit.

    [–] How to get kps upload speed and mbs from a query string? lieutenant_lowercase 2 points ago in learnpython

    You can get total time elapsed by taking current time before and after. Then you can calculate speed by taking the difference in progress and total time elapsed.

    start = datetime.datetime.now()
    ## ADD CALL TO GET PROGRESS ##
    end = datetime.datetime.now()
    time_elapsed = end- start 
    
    >>> time_elapsed.seconds
    5
    >>> time_elapsed .microseconds
    316543
    

    [–] How to format the output of a SQLITE3 query in python? lieutenant_lowercase 2 points ago in learnpython

    for accountname, password in c.fetchall():
        print('Account Name: {} - Password: {}'.format(accountname, password))
    

    [–] Trying to find gaps in a list of dates with CSV, having trouble with tell. lieutenant_lowercase 2 points ago in learnpython

    I would recommend using pandas for this. This code below will load your sample data into a data frame:

    df = pd.read_csv('WN2rQbA9.txt')
    

    then add the 'Date' and 'Time' columns together and convert them to datetime under the new column 'Full_Date'

    df['Full_Date'] = pd.to_datetime(df['Date'] + ' ' + df['Time'])
    

    Next, resample the data hourly. This will leave any missing hours populated but with 'NA' data

    resampled_df = df.set_index('Full_Date')['Air Temp (C)'].resample(rule='1H').sum()
    

    Next, interpolate any missing data. I couldn't really understand your rules, but you can set a limit on how many values to interpolate if you want. docs here . Otherwise you can forward or backfill missing data using .fillna(method='backfill' or 'ffill') docs here

    resampled_df = resampled_df.interpolate(method='linear')
    

    Full code:

    df = pd.read_csv('WN2rQbA9.txt')
    df['Full_Date'] = pd.to_datetime(df['Date'] + ' ' + df['Time'])
    resampled_df = df.set_index('Full_Date')['Air Temp (C)'].resample(rule='1H').sum()
    resampled_df = resampled_df.interpolate(method='linear').reset_index()
    

    Will give you:

    Full_Date Air Temp (C)
    16/06/2009 00:00 5.024
    16/06/2009 01:00 4.74
    16/06/2009 02:00 4.506
    16/06/2009 03:00 4.48
    16/06/2009 04:00 4.48
    16/06/2009 05:00 4.61
    16/06/2009 06:00 4.22
    16/06/2009 07:00 4.141
    16/06/2009 08:00 4.141
    16/06/2009 09:00 4.532
    16/06/2009 10:00 5.437
    16/06/2009 11:00 6.661
    16/06/2009 12:00 7.945
    16/06/2009 13:00 8.742
    16/06/2009 14:00 9.805
    16/06/2009 15:00 10.785
    16/06/2009 16:00 12.775
    16/06/2009 17:00 12.437
    16/06/2009 18:00 12.195
    16/06/2009 19:00 10.98
    16/06/2009 20:00 8.07
    16/06/2009 21:00 6.788
    16/06/2009 22:00 5.924
    16/06/2009 23:00 5.231
    17/06/2009 00:00 4.921
    17/06/2009 01:00 4.921
    17/06/2009 02:00 4.558
    17/06/2009 03:00 3.854
    17/06/2009 04:00 3.459
    17/06/2009 05:00 3.036
    17/06/2009 06:00 2.93
    17/06/2009 07:00 3.248
    17/06/2009 08:00 4.324
    17/06/2009 09:00 6.077
    17/06/2009 10:00 7.594
    17/06/2009 11:00 8.841
    17/06/2009 12:00 10.834
    17/06/2009 13:00 13.305
    17/06/2009 14:00 14.553
    17/06/2009 15:00 15.891
    17/06/2009 16:00 18.295
    17/06/2009 17:00 19.413
    17/06/2009 18:00 17.034
    17/06/2009 19:00 12.147
    17/06/2009 20:00 9.879
    17/06/2009 21:00 8.469
    17/06/2009 22:00 7.494
    17/06/2009 23:00 7.116
    18/06/2009 00:00 6.558
    18/06/2009 01:00 6

    [–] American-style Pizza in London? lieutenant_lowercase 5 points ago in london

    Paradise slice on brick lane is very close