Detecting changes in ICIMS jobs website pages?

[solstyce9]

I have Change Detector hosted locally in a docker container, running every 6 hours to check jobs sites of specific target companies for any newly opened jobs in my area of expertise.

For the most part, it's amazing, thank you!

The one problem I've run into is with the company SAS which uses ICIMS for their job listings. I can load the URL fine in my browser. I look at the screenshot in the history (Chrome browser) and it looks fine, same as in my browser. I have an "extract text" filter of "/.product./" to only show lines that have my target job title in them. However, when I look a the text, it only shows matches for "Product" strings I don't even see in the page. It doesn't match any of the job titles in the page at all.

URL is
https://globalcareers-sas.icims.com/jobs/search?ss=1&searchKeyword=%22product+manager%22&searchRelation=keyword_all

This is a local instance behind my firewall so not accessible for sharing links to the job. If there's a way way to export the settings to include here, please let me know.

Anyone have any experience with this? Any suggestions for ways to get around it?

Screenshots of the history

[geezfools]

I think one of the problems with that site is the careers stuff is "hidden" in an iframe. So changedetection is going to struggle with that as far as my noob brain understands. Guessing you'll need to figure out how to break the iframe out of the site or a way to target the iframe directly.

[solstyce9]

Yeah, I saw the iframe, but was hoping the headless browser would parse that correctly. I tried just loading the iframe URL directly in the browser but it goes back to the same original url.

I was debating seeing if I could try some different headless browsers or config options to get it to work. It definitely doesn't work in the basic plain text browser - the request is blocked.

annoying!

[geezfools]

Yeah, I saw the iframe, but was hoping the headless browser would parse that correctly. I tried just loading the iframe URL directly in the browser but it goes back to the same original url.

I was debating seeing if I could try some different headless browsers or config options to get it to work. It definitely doesn't work in the basic plain text browser - the request is blocked.

annoying!

I monitor icims where I work, just to see what new jobs open up or get filled, though I've been using distill. I figured, hey, I can probably help this guy/gal out. Luckily we don't keep our listings buried in an iframe like that, so it's fairly easy. I'll keep poking around though and see if I can stumble across something before someone with actual skills tells us how to do it correctly :)

[ta7tUHIJHx4RwR21]

https://changedetection.io/share/EdRF99eWZNMa

https://globalcareers-sas.icims.com/jobs/search?ss=1&searchKeyword=%22product+manager%22&searchRelation=keyword_all&in_iframe=1

simple GET request.
copy browser headers from the original website and paste into request headers.
Filters: //*[contains(@Class, 'iCIMS_JobsTable')]

[solstyce9]

Thanks for the tip!

I made sure I was signed into the careers site and then copied the cookies for the job search request and pasted them into the changedetection job. Unfortunately, it didn't make any difference.

When I remove my filters and look at the history, the screenshot shows the jobs like in the above, but there are no jobs appearing in the text history. I'm assuming changedetection.io either can't load or can't see the content in the iframe.

Oddly, the URL in the iframe is the same as the URL in the addressbar, so trying to load the iframe URL doesn't work.

Annoying!

[ta7tUHIJHx4RwR21]

There shouldn't be a screenshot if using a Basic fast Plaintext/HTTP Client fetch method.

[solstyce9]

I've tried it both as plain / basic and chrome / playwright. It seems like it just isn't seeing what's in the iframe.

It makes me wonder if I should be experimenting with the "run JS before detection" to see if I can remove everything that isn't in the iframe.

[ta7tUHIJHx4RwR21]

Have you copied the share link in my post and added it as a watch? It still works for me.

[solstyce9]

Oops. I had tried opening your shared watch link earlier but didn't understand the error. Now that I pasted it into my change detection instance the way I supposed to, it's working great. Thanks!

Now that it's working, I just had to dig deeper to understand what I did wrong. At first I thought it was cookie-related. I was using cookies copied from Firefox. But copying the cookies from your watch didn't fix it. Then I noticed the URL difference - mine didn't have "&in_iframe=1". I added that, and it started working.

Thanks again for all your help!