Chrome Driver - Traverse Iframes & Insert HTML

[NickD]

If would be great if we could have a way of traversing Iframes with Chrome Driver. This is possible with Selenium (which uses ChromeDriver to control Chrome):

Code: Select all

driver.switchTo().frame("//iframe[@src='iframe.html']")

Kantu, a chrome addon that runs a simplified version of Selenium uses the index method of traversing frames, with selectFrame=0 referring to the parent page. Something like that might be a good idea.

I would also like to suggest adding support for execute_script, so that we can inject code into HTML. This is a great feature to be able to fall back on if all attempts at targeting the element fail. eg:

Code: Select all

element = driver.find_element_by_id("#green-widgets")
driver.execute_script("arguments[0].innerHTML = '<button class="static-class-not-dynamic-data-value" onclick="doThis()">Click Here</button>';", element)

[Marcus Tettmar]

We can already switch frames using a custom subroutine which you can see below.

UPDATE: edited to show how to identify frame by attribute

We're aiming to add this as a native function (ChromeSwitchContext?) once we've ironed out a few details and worked out what else everyone wants

This example goes to the contact page on mjtnet.com which hosts the actual form in a frame, so makes a useful example (but please don't spam our contact page by actually submitting anything!). Note the subroutine called SwitchFrame that I have created which you can drop into your own scripts. It takes two parameters - the session ID and the frame index, element object reference json, or null to switch back to the parent window.

Code: Select all

//Start chrome
Let>CHROMEDRIVER_EXE=c:\drive_d\chromedriver.exe
ChromeStart>sessionID

//navigate to mjt's contact form which uses a frame with the fields in it
ChromeNavigate>sessionID,url,https://www.mjtnet.com/contact.htm

//find the frame using its src attribute
ChromeFindElements>sessionID,xpath,//iframe[@src='/contact-form/formpage.html'],elements

//switch browsing context to that frame
GoSub>SwitchFrame,sessionID, {"ELEMENT":"%elements_1%"}

//you can also do it simply by index, zero being the first frame, avoiding using ChromeFindElements if you know its index
//GoSub>SwitchFrame,sessionID,0

//now anything we do is inside that frame so we should be able to get the name field and enter something
ChromeFindElements>sessionID,id,name,inputs
ChromeSetElementValue>sessionID,inputs_1,marcus

ChromeFindElements>sessionID,id,email,inputs
ChromeSetElementValue>sessionID,inputs_1,[email protected]

//we would want to switch back to parent window at some point otherwise subsequent actions will be in the frame
GoSub>SwitchFrame,sessionID,null

ChromeFindElements>sessionID,xpath,//a[@href='/forum/'],forumlinks
ChromeElementAction>sessionID,forumlinks_2,click
Wait>2
ChromeNavigate>sessionID,url,https://www.google.com/

//takes session ID and frame identifier to switch browsing context to
SRT>SwitchFrame
  Let>HTTP_POSTJSON=1
  Let>body= { "id": %SwitchFrame_Var_2% }
  //if you are not using the standard port and have changed it with CHROMEDRIVER_PORT then change the port number here accordingly
  HTTPRequest>http://localhost:9515/session/%SwitchFrame_Var_1%/frame,,POST,body,theResult
END>SwitchFrame

To "traverse" frames you would just keep switching. Each time you switch you change the browsing context. So if there's a frame inside the frame you just switched to just use SwitchFrame again ...

This subroutine should work for the Edge functions too - just change the port to 17556 or whatever you have set EDGEDRIVER_PORT to.

Enjoy!

[Marcus Tettmar]

Updated reply above to show how to identify frames by attributes.

To answer the other point about code injection. Here's another custom subroutine:

Code: Select all

SRT>ExecuteJS
  Let>HTTP_POSTJSON=1
  Let>body= { "script": "%ExecuteJS_Var_2%", "args": [] }
  //if you are not using the standard port and have changed it with CHROMEDRIVER_PORT then change the port number here accordingly
  HTTPRequest>http://localhost:9515/session/%ExecuteJS_Var_1%/execute,,POST,body,theResult
END>ExecuteJS

Call it with .e.g.:

Code: Select all

GoSub>ExecuteJS,sessionID,alert(\"hello\")

So for example after switching frames to the contact form you could set the name field using javascript instead:

Code: Select all

GoSub>ExecuteJS,sessionID,document.getElementById(\"name\").value=\"fred\"

Again, if this is something people find useful we can turn this into a native function.

[Marcus Tettmar]

Well, since we were working on this anyway and it was further ahead than I thought, I managed to reprioritise a few things and get an update pushed out today with added native ChromeSwitchFrame and EdgeSwitchFrame functions. Version 15.0.07. Available in the usual places.

[NickD]

Hi Marcus,

Sorry for the late reply, been busy for the past few days. This is great!

I have a couple of additional questions:

• Is it possible to specify a chrome user profile on ChromeStart? (So I can launch a profile with pre-configured addons etc)

• Is it possible to specify a specific Chrome based browser for ChromeDriver to control? eg:
options.setBinary("C:\\Program Files (x86)\BraveSoftware\Brave-Browser\Application\brave.exe");

[NickD]

Also is there a submit function we can use on form elements? eg:

Code: Select all

ChromeElementAction>session_id,formfield_1,submit

or a sendkeys function:

Code: Select all

ChromeElementAction>session_id,formfield_1,send_keys(Keys.ENTER)

A method of scrolling a specific element would also be very useful:

Code: Select all

ChromeFindElements>session_id,css selector,TabbedContent,Tabs
ChromeElementAction>session_id,Tabs_1,scrollDown,100

[Marcus Tettmar]

Yes ChromeElementAction can take “submit” or “click” actions.

If you specify submit on any form element it should submit the form. See sample script which does that - submits the text box - instead of click the button.

Send keys is exactly what ChromeSetElementValue does.

See docs:

https://www.mjtnet.com/manuals/v15/HTML ... tions.html

Specifying profile/chrome exe not possible at present. Will add those to wish list.

[NickD]

Thanks Marcus,

I edited my last post and added a suggestion for supporting scroll on targeted elements, see above.
(This will be useful for loading data in infinite scroll divs etc)
Will continue to add suggestions if I come across any more features that I think will be beneficial.

[Marcus Tettmar]

Thanks for that.

Regarding sending keystrokes, it may not have been obvious that you can also send non-character keys like Return, Backspace etc. So have updated the docs:

Key Codes for Edge and Chrome
ChromeSetElementValue
EdgeSetElementValue

So e.g.:

Code: Select all

Let>CHROMEDRIVER_EXE=c:\chromedriver.exe

//start a Chrome session
ChromeStart>session_id

//navigate to the form
Let>strURL=https://www.mjtnet.com/contact-form/formpage.html
ChromeNavigate>session_id,url,strURL

//find the message textarea
ChromeFindElements>session_id,name,message,message_elements

//send some text
ChromeSetElementValue>session_id,message_elements_1,First line

//press enter twice
ChromeSetElementValue>session_id,message_elements_1,\uE006
ChromeSetElementValue>session_id,message_elements_1,\uE006

//more text
ChromeSetElementValue>session_id,message_elements_1,Second liKE

//backspace twice
ChromeSetElementValue>session_id,message_elements_1,\uE003
ChromeSetElementValue>session_id,message_elements_1,\uE003

//fixed text
ChromeSetElementValue>session_id,message_elements_1,ne

[NickD]

Thanks Marcus, I did not realise that.

Code: Select all

ChromeElementAction>session_id,chatbox_1,submit

Was not working for me, but:

Code: Select all

ChromeSetElementValue>session_id,chatbox_1,\uE006

Works fine

Does SetElementValue support the same keycodes as Selenium then?

[Marcus Tettmar]

Macro Scheduler is using EdgeDriver/ChromeDriver which are both WebDriver implementations. And the key codes are defined by the WebDriver specs.

[NickD]

Ran into a problem using the code injection SRT you posted due to targeting limitations. The Page I am trying to automate looks like this:

Code: Select all

<html>
<head>
</head>
<body>
<div id="wrap">
<div class="blah blah">...</div>
<div class="blah blah">...</div>
<div class="blah blah">...</div>
<div class="my_target">...</div>
<div class="blah blah">...</div>
</div>
</body>
</html>

I can inject a button like this:

Code: Select all

GoSub>ExecuteJS,session_id,document.getElementById(\"%TargetDiv%\").innerHTML=\"%MyCode%\"

But then I loose all the nested divs which contain content I need to interact with at later stages of the script.

I tried targeting with document.getElementsByClassName and document.getElementsByTagName, but neither worked.

Because the content is dynamic, the number of closing div tags can change so it's not practical to use Regex to pull the div html I need then use StringReplace to insert my code (as I am not sure where the div will end).

Is there a way to replace all the source code at once?, so I could use:

Code: Select all

ChromeGetInfo>session_id,source,theSource
StringReplace>theSource,<body>,<body>%MyCode%,theSource

I would also like to suggest the addition of innerHTML to ChromeGetElementData>:

Code: Select all

ChromeFindElements>session_id,id,TargetDiv,InjectionDiv
ChromeGetElementData>session_id,InjectionDiv_1,innerHTML,InjectionDivHTML

Not only will this be useful when replacing HTML, it will also be very useful for scraping applications where css classes are the only indication of required information.

[Marcus Tettmar]

Your JS will change ALL the html to whatever you have in the MyCode variable.

Unless you have already retrieved the source, added the new html you want appended and concatenated both into MyCode then you will be overwriting everything.

So in your example if you are setting innerHTML of the wrap DIV to just a button then all the other divs and stuff inside the wrap DIV will disappear.

Surely you want to do this:

Code: Select all

document.getElementById(\"%TargetDiv%\").innerHTML=document.getElementById(\"%TargetDiv%\").innerHTML + \"%MyCode%\"

or shorter version using +=

Code: Select all

document.getElementById(\"%TargetDiv%\").innerHTML+=\"%MyCode%\"

E.g.: this adds a paragraph containing an input tag to the end of what is already there ...

Code: Select all

GoSub>ExecuteJS,session_id,document.getElementById(\"wrap\").innerHTML+=\"<p><input type=submit value=try></p>\"

To change entire body source you could do:

Code: Select all

GoSub>ExecuteJS,session_id,document.body.innerHTML=\"<p>entire page replaced with this!</p>\"

[NickD]

Ahaa!

Thanks for the explanation Marcus
Did not realise I was overwriting everything, as the only divs I could target (with an id) were essentially wrapping the whole page anyway.

[NickD]

Still struggling with this...

Code: Select all

GoSub>ExecuteJS,session_id,document.body.innerHTML=\"<p>blah blah</p>\"

Works almost as expected, although instead of replacing the entire page, it actually just replaces everything inside the page body tag.

Here is my code, which uses regex to extract the body tag and insert my desired content (an anchor id and button to scroll to this new anchor, which will hopefully allow me to load all content in an infinite scroll div before scraping the data, analysing, and interacting with it) the modal window displays the modified html as expected, but it does not get injected into the page, any ideas? (Will PM you the URL Marcus).

Code: Select all

  // Get Source HTML
  ChromeGetInfo>session_id,source,pageSource

  // Extract body tag & content
  Let>pattern=<body>.*\s*<\/body>
  RegEx>pattern,pageSource,0,body_array,body_matches,0

  // Verify & Exit if False
  If>body_matches>0
  Let>pageSource=body_array_1
  Else
  If>body_matches=0
  MDL>404 Body not Found!
  Exit>0
  Endif
  Endif


  // Define Custom Button
  Let>MyCode=<div style="width:100%; height:60px; margin:0px auto; position:absolute; top:0px; left:0px;"><button id="custom-btn" onclick="window.location.hash='infinite-loading-container' "; style="padding:14px 16px; background-color:transparent; border:1px solid #ffd300; color:#ffd300;">Injected Button</button></div>
  
  // Regex Replace (Add Button Code to Source)
  Let>pattern=<body>
  RegEx>pattern,pageSource,1,matches,num,1,<body>%MyCode%,pageSource
  
  // Regex Replace (Add Anchor Link ID to Source)
  Let>pattern=<div data-v-358985eb="" class="infinite-loading-container">
  Let>anchorLinkID=<div data-v-358985eb="" class="infinite-loading-container" id="infinite-loading-container">
  RegEx>pattern,pageSource,1,matches,num,1,%anchorLinkID%,pageSource
  
  MDL>pageSource
  
  GoSub>ExecuteJS,session_id,document.body.innerHTML=\"%pageSource%\"