Screen Capture Issue

gchichester · Post by **gchichester** » Fri Dec 26, 2008 12:53 pm

Good Day,
Using the Image Recognition is new to me so please be gentle if Iâ€™m
missing something obvious.
When I run the script below It seems all work except the â€œLClickâ€

Bob Hansen · Post by **Bob Hansen** » Fri Dec 26, 2008 6:36 pm

The log file seems to indicate that "imgs" was probably = 0.

Single step in troubleshoot mode, or insert a Breakpoint, and look at the value for the variable "imgs". If value is not > 0 then left click will not execute.

Two comments about logs:
1. Use % around your variables in formulas and you will be able to see the values in the log files. In your example:

Code: Select all

WaitWindowOpen>Export Manager - [E0HSRG: Containers / Purchase Orders / Com. Invoices / On-Hand Info]
Wait>%vWait%
GetScreenRes>sX,sY
ScreenCapture>0,0,%sX%,%sY%,c:\CP_Macros\Screen.bmp
FindImagePos>C:\CP_Macros\ContainerNo.bmp,c:\CP_Macros\screen.bmp,20,1,xPos,yPos,imgs
Wait>%vWait%
If>%imgs%>0
MouseMove>%xPos_0%,%yPos_0%
LClick
EndIf

2. Rather than having the log show both before and after, just select the second option, before running a step.

gchichester · Post by **gchichester** » Sat Dec 27, 2008 9:49 pm

Thanks Bob,
I used your suggestion and added the % to my log and low and behold
you were right "imgs" is coming back with a 1 and not a 0.
So does that mean the "FindImagePos" is not working?
Being new to the Image Recognition option I'm sure the best way to
troubleshoot this problem.

Thanks again for you help

gchichester

Bob Hansen · Post by **Bob Hansen** » Sun Dec 28, 2008 12:05 am

The first log acted as if imgs was <=0. It imgs =1, then the mouse move/click should have worked. You are saying imgs =1, so the log must be different.

Can you print the log again?

gchichester · Post by **gchichester** » Sun Dec 28, 2008 3:02 pm

Bob,
Well for some reason this time when I ran the script "imgs" is coming back with a "0". As seen in the log, and still no mouse movment.

I'm not sure if it matter but I'm running this in a terminal server session
via a rdp connection. Also I tried the sample script FindImagePos Test
and it doesn't work ether, l included that log as well.
I Hope this helps
gchichester

12/28/2008 09:33:45:670 - Started Macro : EditNewBill
12/28/2008 09:33:45:686 - LET>IGNORESPACES=1
12/28/2008 09:33:45:686 - LET>vWait=0.25
12/28/2008 09:33:45:686 - /*
12/28/2008 09:33:45:701 - WaitKeyDown>VK123
12/28/2008 09:33:49:201 - Press Tab * 4
12/28/2008 09:33:49:264 - Wait>vWait
12/28/2008 09:33:49:514 - WaitKeyDown>VK123
12/28/2008 09:33:50:264 - Wait>vWait
12/28/2008 09:33:50:514 - Press ALT
12/28/2008 09:33:50:529 - Send>p
12/28/2008 09:33:50:529 - Release ALT
12/28/2008 09:33:50:529 - WaitWindowOpen>Export Manager - [E0HSRG: Containers / Purchase Orders / Com. Invoices / On-Hand Info]
12/28/2008 09:33:50:545 - Wait>vWait
12/28/2008 09:33:50:795 - GetScreenRes>sX,sY
12/28/2008 09:33:50:795 - ScreenCapture>0,0,1024,768,c:\CP_Macros\Screen.bmp
12/28/2008 09:33:50:904 - FindImagePos>C:\CP_Macros\ContainerNo.bmp,c:\CP_Macros\screen.bmp,20,1,xPos,yPos,imgs
12/28/2008 09:33:51:061 - Wait>vWait
12/28/2008 09:33:51:311 - If>0>0
12/28/2008 09:33:51:311 - /*
12/28/2008 09:33:51:311 - Finished Macro : EditNewBill

12/28/2008 09:44:45:857 - Started Macro : FindImagePos Test
12/28/2008 09:44:45:873 - /*
12/28/2008 09:44:45:873 -
12/28/2008 09:44:45:873 - VBSTART
12/28/2008 09:44:45:889 -
12/28/2008 09:44:45:889 - Dialog>Dialog1
12/28/2008 09:44:45:904 -
12/28/2008 09:44:45:904 - Show>Dialog1,r
12/28/2008 09:44:54:842 -
12/28/2008 09:44:54:842 - If>r=10
12/28/2008 09:44:54:842 - VBEval>Timer,startTime
12/28/2008 09:44:54:857 - FindImagePos>Dialog1.msEdit1,SCREEN,Dialog1.msEdit2,1,X,Y,n
12/28/2008 09:44:54:904 - If>n>0
12/28/2008 09:44:54:904 - VBEval>Timer-35094.85546875,Elapsed
12/28/2008 09:44:54:920 - MessageModal>Seconds Elapsed: 0.06640625
12/28/2008 09:44:57:404 - Endif
12/28/2008 09:44:57:420 -
12/28/2008 09:44:57:420 -
12/28/2008 09:44:57:420 - Finished Macro : FindImagePos Test

Bob Hansen · Post by **Bob Hansen** » Sun Dec 28, 2008 3:41 pm

So, the problem is not that mouse is not moving and clicking, that is just the symptom. The problem is that the image is not being found. This line is sending "0" to imgs: FindImagePos>C:\CP_Macros\ContainerNo.bmp,c:\CP_Macros\screen.bmp,20,1,xPos,yPos,imgs

Where/How is C:\CP_Macros\ContainerNo.bmp defined? Are you sure that it exists? Could do IfFileExists for both images before doing FindImagePos command.

gdyvig · Post by **gdyvig** » Sun Dec 28, 2008 6:11 pm

The fact you are using a termserv session may be a significant clue. The safest way is to capture your images and run your scripts under exactly the same configuration. When your script runs, have it do a ScreenCapture to a bmp file. Are both it and your needle file the same color depth (16, 24, or 32). If they are different, you may have image recognition problems. Changing the Window display settings on your local desktop will change the color depth on your remote desktop, sometimes in surprising ways.

Gale

Aaron · Post by **Aaron** » Sun Dec 28, 2008 7:18 pm

Code: Select all

WaitWindowOpen>Export Manager - [E0HSRG: Containers / Purchase Orders / Com. Invoices / On-Hand Info]
Wait>1
GetScreenRes>sX,sY
ScreenCapture>0,0,%sX%,%sY%,c:\CP_Macros\Screen.bmp
FindImagePos>c:\CP_Macros\ContainerNo.bmp,c:\CP_Macros\Screen.bmp,20,1,xPos,yPos,imgs
Wait>2
If>%imgs%>0
MouseMove>%xPos_0%,%yPos_0%
MessageModal>should work, try insurting wait=1

EndIf
MessageModal>No img found, make sure you close Export Manager then try again

Bob Hansen · Post by **Bob Hansen** » Sun Dec 28, 2008 8:16 pm

The last message will run under all conditions.
You only want it to run if no image found. Use ELSE vs. EndIf

Code: Select all

Else
MessageModal>No img found, make sure you close Export Manager then try again
EndIf

gchichester · Post by **gchichester** » Sun Dec 28, 2008 8:54 pm

Bob,
Yes I have confirmed that the screen.bmp and ContainerNo.bmp files
do exist in the CP_Macros folder and I'm able to view them.

Gale,
I'm running the script from the same session each time, the script does
include a ScreenCapture that runs each time and I have verified that
the .bmp is new after is runs, So my thinking is because I'm getting a new
file each time that the color depth should not matter, but I checked and
both my desktop and the rdp session are set for 16, which is the only
option I can use.

It seems to me that theres some other issue here because as I stated
the sample script FindImagePos Test doesn't work ether.
So if it can't find the image I just captured then adding the MessageModal to my script won't tell me anything new.

Thanks to all for you feedback
gchichester

Me_again · Post by **Me_again** » Sun Dec 28, 2008 9:41 pm

I'm voting with Gale on this one, most likely some subtle difference in the rdp display rendering. How did you capture the ContainerNo.bmp? Was it using MacroScheduler on the PC running the macro, and from an rdp desktop of the remote PC?

gdyvig · Post by **gdyvig** » Sun Dec 28, 2008 9:48 pm

Another thing that sometimes causes problems is dithering or a dynamic background.

Dithering can occur when a 24 or 32bit image is displayed at 16bit Windows settings. Increasing the color tolerance helps.

Does your ContainerNo needle file contain a little bit of unpredictable screen background? Increasing the color tolerance helps here too, but eliminating the background from your needle file works best.

Gale

gchichester · Post by **gchichester** » Mon Dec 29, 2008 3:57 pm

Gale,
How can you tell if there's a dithering problem? I checked my rdp config and it's set for 16 bit the same as the terminal server,
And as far as I can tell this canâ€™t be changed, I mean the rdp session defaults to 16 bit no matter what I set the rdp config to or the terminal server desktop.
Not that I can tell - For testing I'm using a button from MS Word.
Macro Scheduler 11 has been installed on my Terminal Server from the start, where I created all my scripts and screen captures from a rdp session.
This is the server thatâ€™s running the application Iâ€™m trying to automate.
Also I tried matching the screen resolution on both the Terminal Server and my rdp session, didnâ€™t seem to help.
I was thinking that if the script is running a â€œGetScreenSesâ€

gdyvig · Post by **gdyvig** » Mon Dec 29, 2008 6:31 pm

I think there is a command called GetPixelColor that will let you get the color codes for individual pixels. If adjacent pixels in what appears to be a solid area have different pixel colors, you may have dithering. If you do the same experiment on a machine set to 24 or 32bit color and all the pixels in the same solid colored area have the same color code, you do not have dithering.

Usually the dithered colors are very similar and may be difficult to distinguish. That is why increasing the color tolerance works.

Gale

JRL · Post by **JRL** » Mon Dec 29, 2008 11:36 pm

This script requires Macro Scheduler version 11.

Here's a cheap and dirty way to check for dithering. Put the image name in the second line of the script under the "ImageName" variable. Unremark one of the three lines that sets the background color for the background dialog. If needed change the size or location variables for the dialogs (too big is better than too small).

Start the script and you should see your image displayed. Hold the CTRL key and select a pixel using the left mouse button. the color of the selected pixel in the image should become transparent and display the color of the background dialog. Winkey+Esc should close the script. If you pick on what appears to be a solid color and it doesn't all turn to the color of the background dialog, you have a dithering issue.

Hopefully this makes sense.
Later,
Dick

Code: Select all

//Set the image name
Let>ImageName=drive:\path\image.bmp or .jpg

//Set background color to an appropriate primary color
//For primary red
RGB>255,0,0,BackColor
//For primary green
//RGB>0,255,0,BackColor
//For primary blue
//RGB>0,0,255,BackColor
//For black
//RGB>0,0,0,BackColor
//For white
//RGB>255,255,255,BackColor

//set size and location of the dialogs
Let>Wide=700
Let>Tall=500
Let>XLoc=0
Let>YLoc=0


//TransparentDialog usage
//GoSub>TransparentDialog,DialogName,TransparentColorNumber
SRT>TransparentDialog
Let>dlg_hndl=%TransparentDialog_var_1%.handle
//constants
  Let>GWL_EXSTYLE=-20
  Let>WS_EX_LAYERED=524288
  Let>LWA_COLORKEY=1
//get style attributes of dialog1 window
  LibFunc>user32,GetWindowLongA,attribs,dlg_hndl,GWL_EXSTYLE
  Let>attribs={%attribs% OR %WS_EX_LAYERED%}
//make selected color transparent
  LibFunc>user32,SetWindowLongA,swl,dlg_hndl,GWL_EXSTYLE,attribs
  LibFunc>user32,SetLayeredWindowAttributes,res,dlg_hndl,%TransparentDialog_var_2%,0,LWA_COLORKEY
END>TransparentDialog

OnEvent>key_down,VK1,2,ShowRed
OnEvent>key_down,VK27,8,Quit

SRT>ShowRed
  GetCursorPos>CurX,CurY
  GetPixelColor>CurX,CurY,PixCol
  GoSub>TransparentDialog,Dialog1,PixCol
  Wait>1
END>ShowRed

SRT>Quit
  Exit>0
END>Quit

Dialog>Dialog1
   Caption=Image
   Width=%Wide%
   Height=%Tall%
   Top=%XLoc%
   Left=%YLoc%
   Image=%ImageName%,0,0,%Wide%,%Tall%
EndDialog>Dialog1

Dialog>Dialog2
   Caption=Color
   Width=%Wide%
   Height=%Tall%
   Top=%XLoc%
   Left=%YLoc%
EndDialog>Dialog2

Show>Dialog2
Show>Dialog1

SetDialogObjectColor>Dialog2,,BackColor

Label>Loop
  Wait>0.01
Goto>Loop

Screen Capture Issue

Screen Capture Issue

termserv sessions affect color depth

just run this script and let us know what happens

Background and dithering

GetPixelColor