@imfm
I need to automate pulling data from several websites with atrocious autogenerated spaghetti code. I was trying with Rvest alone and httr and other solutions. I was getting nowhere fast. Then I found this video and boom, I'm in. I can't thank you enough Samer.
@michaelhudson2375
Excellent video Samer. Thank you very much for sharing such useful content. I never thought I would be able to use RSelenium to scrape a site but following your instructions I was able to get a table and a list that I need for work. I look forward to working through your other tutorials. All the best to you.
N/A
Very well explained. I didn't' know about {RSelenium}, looks really powerful. Thanks!
@RolandŐzse
Hi, Thank you so much for this. I am not that big on coding and this solution is really easy to follow. Excuse me if I am being too dumb. I ran into a problem when you refer to the pagination command at 5:25 using the aria label. I am trying to scrape a transfermarkt table and that field is looking pretty different for me: <a href="/spieler-statistik/marktwertspruenge/marktwertetop?ajax=yw1&page=2" title="Go to next page" class="tm-pagination__link"> </a> As you can see, it's a href and not an aria label. There is a link to the next page on every page and I do not know how to iterate this. Works fine if I want to do the first two page but then It's obviously not working. Could you maybe help me out what I should copy paste to the findElement function? Or is this a whole different situation and I have to do something new? Thank you for your help in advance :)
@AliRami07
I loved this tutorial! Quick question, do you think HasData's API could simplify the process of scraping dynamic tables like the ones shown here?
@Franciszek-uy4kz
Quick question, do you think using HasData's API could help streamline the scraping process for dynamic tables like those in this video?
@huongheidinguyen337
Thank you for the tutorial. I'm practicing scraping Sephora product reviews and ran into a problem. On my last page, there is still a Next page button (it is just disabled), so there was no error and my Next-page loop didn't end. Do you have any suggestions on how to end the loop in this case?
@arunrajesh5137
Watching this tutorial immediately after your Introduction to RSelenium. Really enjoyed learning it from you Samer. How do we navigate to a webpage with username and password from RSelenium ?
@eleonoras.2878
Thank you very much for providing such a great explanation! I've encountered an issue in that I'm only seeing a limited selection of chromedriver versions. Unfortunately, none of these versions seem to be compatible with my current Google Chrome version. Would you by any chance have any suggestions on how I might go about resolving this problem? Your insights would be greatly appreciated. :)
@sarahsuzz
I keep getting an error "element not found" when using xpath to locate my "nextpage" button - it is an aria-label and it's located in the div section of the DOM - not sure what I am doing wrong. I have checked my code for typos, very carefully. Can you help?
@馮庭萱
many thanks. great explaination, super clear !
@AngelFelizF
Great video, thanks for sharing
@devypratiwi8103
hello thanks for sharing the video! so i've already watched and followed all the steps but i got an error saying Error in java_check() : PATH to JAVA not found. Please check JAVA is installed. but something that makes confuses is i've also already installed my JAVA till it complete but the error keeps saying that JAVA is not found. Do you know how to solve this issue? thankyou
@delabungsu6817
Thank you Samer.
@shoakromyusupov7297
Really helpful video. Would like to ask if you can make similar video to scrape data from social media sites like Instagram, LinkedIn or from your own preference ?
@НикитаБабарыкин-р1ь
Can you help please? Error in checkError(res) : Undefined error in httr call. httr output: Failed to connect to localhost port 4567 after 2254 ms: Connection refused What can be a problem?
@cameronl1434
Sorry I am very much a beginner with all this so sorry if this is a stupid question. I have a data table which I want to extract the information from but when I inspect the code it doesn't have an ID. How can I go about selecting the date table without an ID? Thank you in advance
@CreativeOutput_L_Rist
Hey Samer, love the tutorial but ran into an issue I couldn't resolve yet. I am using RSelenium to click on a tab that contains the data I want, which works fine if I run the lines of code one after the other, but not in a for loop. I have a list of links the loop should iterate through and some tries it didn't even click the tab for the first list item, other times it stopped after just a couple.. after just adding a bunch of clickElement() commands it worked for a bit longer (but not directly related to the number of commands added) and then stopped again. Any idea how to make it run more stable? My R memory usage is kinda high, could it be due to that? Am a total noob at R, but confusing that it works manually but not in the loop Edit: Also, the netstat free_port function always gives me an 'Error in strsplit(local, ":") : non-character argument'.. I wrote it exactly as you have, so no idea why it doesn't work.. if I define a port manually it (e.g. 14415 or '14415') it says 'Error: port should be an integer value'.. my knowledge of maths might be limited but last time I checked 14415 was an integer lol
@yehitzmedapirc
Hi! What can I do if I my "Next button" is different every time? I do not have a "next" button, I have ti click on the 1, then 2 etc on the page. Thanks!
@remingtonsavage9504