Scrape Table Where Rows Are Distributed Between Many Tr

I want to scrape data from this website table: https://www.oddsportal.com/moving-margins/ This is the code I used: It return nested lists, to put them as rows on a csv file, but th

Solution 1:

You can use below code to get all the row (with no class and with odd class) associated with with row with dark class.

u ='https://www.oddsportal.com/moving-margins/'

driver.maximize_window()
driver.get(u)
#Use Explicit time wait for fast execution
WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#moving_margins_content_overall")))
driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
table_data =  driver.find_elements_by_xpath("//div[@id='moving_margins_content_overall']//tr[@class='dark']")
#driver.find_element_by_id("").is_displayed()
table=[]
# Creating a list of lists, whereeach list consist all data ineachrow either with class dark or odd
for data in table_data:
    row= []
    # toget data in dark tr
    dark_row = data.find_elements_by_xpath((".//th//a"))
    for col in dark_row:
        row.append(col.text.replace("\n"," "))

    #Togetall the rows related to above dark row
    blank_rows = data.find_elements_by_xpath(".//following-sibling::tr//th[1]")
    odd_rows = data.find_elements_by_xpath(".//following-sibling::tr[@class='odd']")
    for blank, odd in zip(blank_rows, odd_rows):
        row.append(blank.text) # Toget data from th with first2 class
        odd_data = odd.find_elements_by_xpath(".//td")
        for od in odd_data:
            row.append(od.text.replace("\n"," ")) # getall data fromrowwith odd class
        row.append(odd_data[-1].find_element_by_xpath('.//a').get_attribute("title")) # get bookmaker title for odd row
    table.append(row)
for t intable:
    print(t)

Output: As you can see there are 7 associated odd rows (and row with no class ) for dark row showing Rugby Union match. Ans data is in same list along with bookmaker title for each odd.