<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Martin Petrov</title>
    <description>The latest articles on DEV Community by Martin Petrov (@martinpython).</description>
    <link>https://dev.to/martinpython</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4229221%2F1b5cc3a9-84c9-47c8-8bb6-f4f74730fc15.webp</url>
      <title>DEV Community: Martin Petrov</title>
      <link>https://dev.to/martinpython</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/martinpython"/>
    <language>en</language>
    <item>
      <title>Validate IBANs in Python with the mod-97 checksum (and what it can't catch)</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Wed, 18 Nov 2026 10:25:12 +0000</pubDate>
      <link>https://dev.to/martinpython/validate-ibans-in-python-with-the-mod-97-checksum-and-what-it-cant-catch-2h72</link>
      <guid>https://dev.to/martinpython/validate-ibans-in-python-with-the-mod-97-checksum-and-what-it-cant-catch-2h72</guid>
      <description>&lt;p&gt;Every IBAN carries its own checksum: two digits right after the country code. You can catch almost every typo in a bank account number with about ten lines of Python, before a payment bounces and the bank charges you a fee.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the IBAN check works (ISO 13616, mod 97)
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Remove spaces and make the letters upper case.&lt;/li&gt;
&lt;li&gt;Check the length for the country (DE = 22, FR = 27, NL = 18, BG = 22, ES = 24, IT = 27, GB = 22).&lt;/li&gt;
&lt;li&gt;Move the first 4 characters (country code + check digits) to the end.&lt;/li&gt;
&lt;li&gt;Replace every letter with two digits: A = 10, B = 11 ... Z = 35.&lt;/li&gt;
&lt;li&gt;Read the result as one big number. If &lt;code&gt;number % 97 == 1&lt;/code&gt;, the IBAN is valid.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Python has arbitrary-size integers, so step 5 is one line.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="no"&gt;LENGTHS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"DE"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"FR"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;27&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"NL"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;18&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"BG"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ES"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;24&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"IT"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;27&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
           &lt;span class="s2"&gt;"GB"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"AT"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"BE"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"CH"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"PL"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;28&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"RO"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;24&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;iban_ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;iban&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;iban&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;" "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;upper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isalnum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"only letters and digits are allowed"&lt;/span&gt;
    &lt;span class="n"&gt;want&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="no"&gt;LENGTHS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;want&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;want&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{s[:2]} IBANs have {want} characters, this one has {len(s)}"&lt;/span&gt;
    &lt;span class="n"&gt;moved&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;digits&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;36&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;moved&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;# int("A", 36) == 10&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;digits&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="mi"&gt;97&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"checksum wrong: probably a typo"&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ok"&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"DE89 3704 0044 0532 0130 00"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;   &lt;span class="c1"&gt;# documented example IBAN&lt;/span&gt;
          &lt;span class="s2"&gt;"DE89 3704 0044 0532 0130 01"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;   &lt;span class="c1"&gt;# one digit changed&lt;/span&gt;
          &lt;span class="s2"&gt;"GB82 WEST 1234 5698 7654 32"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  &lt;span class="c1"&gt;# documented example IBAN&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;iban_ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;DE89 3704 0044 0532 0130 00 (True, 'ok')
DE89 3704 0044 0532 0130 01 (False, 'checksum wrong: probably a typo')
GB82 WEST 1234 5698 7654 32 (True, 'ok')
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Check a whole list from a CSV
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;

&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"suppliers.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;newline&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"utf-8"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; \
     &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"suppliers_checked.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;newline&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"utf-8"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;DictReader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;DictWriter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fieldnames&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fieldnames&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"iban_ok"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"iban_note"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writeheader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;note&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;iban_ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"iban"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writerow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"iban_ok"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"iban_note"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;note&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What mod 97 does NOT tell you
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It does not prove the account exists.&lt;/strong&gt; It only proves the string is well formed. A closed account still passes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Some countries add a national check digit&lt;/strong&gt; inside the BBAN (for example France, Italy, Spain, Belgium). A full validator checks that too.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The bank&lt;/strong&gt; is encoded in the BBAN part (German Bankleitzahl, UK sort code). To show the bank name you need a bank directory, not just maths.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Common mistakes I see in payment files
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The letter O typed instead of zero, or I instead of 1. &lt;code&gt;isalnum()&lt;/code&gt; passes them, the checksum catches them.&lt;/li&gt;
&lt;li&gt;Copying from a PDF adds non-breaking spaces (&lt;code&gt;&amp;nbsp;&lt;/code&gt;). Replace them too: &lt;code&gt;iban.replace("&amp;nbsp;", "")&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Excel turns long numbers into &lt;code&gt;3.70400440532013E+17&lt;/code&gt; when an account number column is formatted as a number. Format the column as text before you paste.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  No code needed
&lt;/h2&gt;

&lt;p&gt;If you only need to check one IBAN, or you want to send the check to a colleague who does not write Python, the free &lt;a href="https://radarbg.com/tools/iban-checker?ref=devto" rel="noopener noreferrer"&gt;IBAN checker&lt;/a&gt; does the same checks, including the national check digit, and shows the bank. There is also a page per country with the exact layout, for example &lt;a href="https://radarbg.com/iban?ref=devto" rel="noopener noreferrer"&gt;German IBAN format&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What else do you validate before you send a payment file?&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>fintech</category>
      <category>tutorial</category>
      <category>beginners</category>
    </item>
    <item>
      <title>Find resellers selling below your MAP price every day with Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Sat, 14 Nov 2026 22:05:05 +0000</pubDate>
      <link>https://dev.to/martinpython/find-resellers-selling-below-your-map-price-every-day-with-python-4hi1</link>
      <guid>https://dev.to/martinpython/find-resellers-selling-below-your-map-price-every-day-with-python-4hi1</guid>
      <description>&lt;p&gt;If you sell a product through other shops, you need to know which of them list it below your minimum advertised price (MAP), and you need to know it the day it happens, not at the end of the month.&lt;/p&gt;

&lt;p&gt;This is part 3 of a small series. In &lt;a href="https://dev.to/martinpython/compare-one-products-price-across-online-stores-with-python-real-output-3dd5"&gt;part 1&lt;/a&gt; I compared one product's price across stores, and in &lt;a href="https://dev.to/martinpython/get-only-the-price-changes-from-online-stores-every-day-with-python-5595"&gt;part 2&lt;/a&gt; I added a change mode that returns only offers whose price or stock moved. This post adds the MAP check I built on top of it this week: give it your MAP and it flags every offer below it, with the gap in money and in percent. Real output from today (2026-11-14) is below, plus the limits.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;Most small brands check MAP like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Someone opens 20 to 200 reseller pages in a browser, once a week if there is time.&lt;/li&gt;
&lt;li&gt;They copy prices into a spreadsheet, some in USD, some in EUR or GBP.&lt;/li&gt;
&lt;li&gt;They compare each one with the MAP column by hand.&lt;/li&gt;
&lt;li&gt;A reseller drops the price on Monday, and the brand finds out on Friday, after other resellers have already matched it.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A script helps, but then you own the scraper, the currency conversion, the per-product MAP table and the "what changed since yesterday" logic. That last part is the one that breaks.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the tool does
&lt;/h2&gt;

&lt;p&gt;I built the Cheapest Price Finder on Apify. You give it product page URLs from any stores. It reads the real price from each page's structured data (JSON-LD offers, Shopify product data, meta price tags), converts everything to one currency with daily ECB rates and sorts the offers from cheapest to most expensive.&lt;/p&gt;

&lt;p&gt;The MAP check adds three inputs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;mapPrice&lt;/code&gt;: your MAP, in the &lt;code&gt;convertTo&lt;/code&gt; currency.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;mapPrices&lt;/code&gt; (optional): a different MAP per product URL, as &lt;code&gt;{url: price}&lt;/code&gt;. It overrides &lt;code&gt;mapPrice&lt;/code&gt; for that URL.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;onlyChanged&lt;/code&gt; + &lt;code&gt;monitorId&lt;/code&gt; (from part 2): return only new or changed offers, so a daily run is quiet unless something moved.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every row then gets &lt;code&gt;mapPrice&lt;/code&gt;, &lt;code&gt;belowMap&lt;/code&gt; (true/false), &lt;code&gt;mapGap&lt;/code&gt; (how far below MAP, in money; negative means above MAP) and &lt;code&gt;mapGapPercent&lt;/code&gt;. &lt;code&gt;RUN_SUMMARY.mapCheck&lt;/code&gt; lists &lt;code&gt;belowMapCount&lt;/code&gt; and &lt;code&gt;belowMapUrls&lt;/code&gt;, so one key tells you whether today needs attention.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get a token
&lt;/h2&gt;

&lt;p&gt;Create a free Apify account and copy your token from Console (Settings, API and Integrations):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;your_token_here
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: one run with curl
&lt;/h2&gt;

&lt;p&gt;For the demo I used the AeroPress Original at four coffee shops. I am not the brand and I do not know AeroPress's real MAP, so I picked a made-up MAP of $42, and for one store a separate MAP of $49.95 to show the per-URL override:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~cheapest-price-finder/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"productUrls": [
        "https://www.vervecoffee.com/products/aeropress-coffee-brewer",
        "https://www.intelligentsia.com/products/aeropress",
        "https://www.espressoparts.com/products/aeropress-coffee-maker",
        "https://www.stumptowncoffee.com/products/aeropress"],
       "convertTo": "USD",
       "mapPrice": 42,
       "mapPrices": {"https://www.stumptowncoffee.com/products/aeropress": 49.95}}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Real output (2026-11-14, run took 4.6 s)
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;store&lt;/th&gt;
&lt;th&gt;price&lt;/th&gt;
&lt;th&gt;availability&lt;/th&gt;
&lt;th&gt;mapPrice&lt;/th&gt;
&lt;th&gt;belowMap&lt;/th&gt;
&lt;th&gt;mapGap&lt;/th&gt;
&lt;th&gt;mapGapPercent&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;vervecoffee.com&lt;/td&gt;
&lt;td&gt;39.95 USD&lt;/td&gt;
&lt;td&gt;InStock&lt;/td&gt;
&lt;td&gt;42&lt;/td&gt;
&lt;td&gt;true&lt;/td&gt;
&lt;td&gt;2.05&lt;/td&gt;
&lt;td&gt;4.9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;intelligentsia.com&lt;/td&gt;
&lt;td&gt;40.00 USD&lt;/td&gt;
&lt;td&gt;OutOfStock&lt;/td&gt;
&lt;td&gt;42&lt;/td&gt;
&lt;td&gt;true&lt;/td&gt;
&lt;td&gt;2.00&lt;/td&gt;
&lt;td&gt;4.8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;espressoparts.com&lt;/td&gt;
&lt;td&gt;44.95 USD&lt;/td&gt;
&lt;td&gt;InStock&lt;/td&gt;
&lt;td&gt;42&lt;/td&gt;
&lt;td&gt;false&lt;/td&gt;
&lt;td&gt;-2.95&lt;/td&gt;
&lt;td&gt;-7.0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;stumptowncoffee.com&lt;/td&gt;
&lt;td&gt;46.00 USD&lt;/td&gt;
&lt;td&gt;InStock&lt;/td&gt;
&lt;td&gt;49.95&lt;/td&gt;
&lt;td&gt;true&lt;/td&gt;
&lt;td&gt;3.95&lt;/td&gt;
&lt;td&gt;7.9&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;And the summary from the same run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"mapCheck"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"belowMapCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"belowMapUrls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.vervecoffee.com/products/aeropress-coffee-brewer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.intelligentsia.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.stumptowncoffee.com/products/aeropress"&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"mapPrice"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;42.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"perUrlPrices"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"currency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"USD"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note the Intelligentsia row: it is below MAP but out of stock. Whether that counts as a violation is your policy, so the tool reports &lt;code&gt;availability&lt;/code&gt; and leaves the decision to you.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: the same in Python, as a daily watch
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;ACTOR&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"martin10291~cheapest-price-finder"&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items"&lt;/span&gt;

&lt;span class="no"&gt;RESELLERS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.vervecoffee.com/products/aeropress-coffee-brewer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.intelligentsia.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.espressoparts.com/products/aeropress-coffee-maker"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.stumptowncoffee.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;map_violations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;map_price&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;per_url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"productUrls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;RESELLERS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"convertTo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"USD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"mapPrice"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;map_price&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"mapPrices"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;per_url&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"onlyChanged"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;          &lt;span class="c1"&gt;# only new or changed offers&lt;/span&gt;
        &lt;span class="s2"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"aeropress-map"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="c1"&gt;# keeps yesterday's prices for you&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                      &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"belowMap"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"__main__"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;map_violations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;42&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'{row["store"]}: {row["price"]} {row["currency"]} '&lt;/span&gt;
              &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'({row["mapGapPercent"]}% below MAP, {row["availability"]}, '&lt;/span&gt;
              &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'change: {row["changeType"]})'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With &lt;code&gt;onlyChanged&lt;/code&gt;, the first run returns every offer once as &lt;code&gt;new&lt;/code&gt;. I ran it twice today with MAP $39.95 and the same &lt;code&gt;monitorId&lt;/code&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Run 1 (6.5 s): 4 rows, all &lt;code&gt;changeType: "new"&lt;/code&gt;, &lt;code&gt;belowMapCount: 0&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Run 2 (5.2 s): 0 rows saved, summary &lt;code&gt;"unchanged": 4&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So on a normal day your script prints nothing. On the day a reseller cuts the price, you get one row with &lt;code&gt;changeType: "priceDrop"&lt;/code&gt;, &lt;code&gt;previousPrice&lt;/code&gt;, &lt;code&gt;priceChange&lt;/code&gt; and the MAP gap together.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: schedule it
&lt;/h2&gt;

&lt;p&gt;The simplest way, no server:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open the &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;actor page&lt;/a&gt;, paste your reseller URLs, set &lt;code&gt;mapPrice&lt;/code&gt;, tick "Only changed offers" and type a Monitor ID.&lt;/li&gt;
&lt;li&gt;Save it as a Task and add a schedule, e.g. every day at 07:00.&lt;/li&gt;
&lt;li&gt;Optional: connect the dataset to Slack, email or Google Sheets with Apify integrations, or add a webhook on "run succeeded".&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Or run the Python file from your own cron:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;0 7 * * * /usr/bin/python3 /home/me/map_watch.py &amp;gt;&amp;gt; /home/me/map.log 2&amp;gt;&amp;amp;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Without an Apify account
&lt;/h2&gt;

&lt;p&gt;The MAP check (not the change mode) also works through plain HTTP APIs with one key:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;API.market&lt;/a&gt;, free plan to test.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://radarbg.com/apis/cheapest-price-finder" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;, my own site, 100 free calls a month with one key for all my APIs.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST https://radarbg.com/v1/cheapest-price-finder \
  -H &lt;span class="s1"&gt;'X-API-Key: YOUR_KEY'&lt;/span&gt; -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"productUrls": ["https://www.vervecoffee.com/products/aeropress-coffee-brewer",
                      "https://www.espressoparts.com/products/aeropress-coffee-maker"],
       "convertTo": "USD", "mapPrice": 42}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I tested this path today too: the same two rows came back with &lt;code&gt;belowMap&lt;/code&gt; true and false. &lt;code&gt;onlyChanged&lt;/code&gt; and &lt;code&gt;monitorId&lt;/code&gt; are switched off on these APIs on purpose: the remembered prices would live in my account and be shared by every caller with the same URLs. Keep yesterday's rows yourself there, or use the Apify actor for the built-in change feed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Big marketplaces often block automated requests.&lt;/strong&gt; Amazon, Walmart and Best Buy frequently answer with a bot-check page. The tool does not bypass that; such pages land in &lt;code&gt;RUN_SUMMARY.failed&lt;/code&gt; and are not charged. Brand stores and independent shops (Shopify, WooCommerce, Magento, BigCommerce) work best.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It reads the listed price only&lt;/strong&gt;: no coupons, cart discounts or shipping. A reseller who hides the real price in the cart will not show up.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You choose the URLs.&lt;/strong&gt; It does not find your resellers for you or match products; &lt;code&gt;gtin&lt;/code&gt;, &lt;code&gt;sku&lt;/code&gt; and &lt;code&gt;brand&lt;/code&gt; in each row help you check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pages that build the price only with JavaScript&lt;/strong&gt; and publish no product data return "no price found".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It does not contact anyone.&lt;/strong&gt; It tells you who is below MAP; what you do next is up to you.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;On Apify it costs $3.00 per 1,000 saved results. With &lt;code&gt;onlyChanged&lt;/code&gt;, unchanged offers are not saved and not billed, so 50 resellers checked daily cost nothing for results on a quiet day. Apify's free plan includes a small monthly credit, enough to try it. On API.market and radarbg.com there are free plans to test first.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because MAP checks are the same price scrape plus one comparison, and the comparison is what people end up doing by hand. Now one field does it.&lt;/p&gt;

&lt;p&gt;If you manage a brand with resellers, try it on &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;Apify&lt;/a&gt; with 3 to 5 of your reseller pages. How many reseller pages would your daily list have? If a page returns "no price found", post the URL in the comments and I will look at it.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>ecommerce</category>
      <category>automation</category>
    </item>
    <item>
      <title>Check a list of VINs for new NHTSA recalls every day with Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Fri, 13 Nov 2026 22:05:03 +0000</pubDate>
      <link>https://dev.to/martinpython/check-a-list-of-vins-for-new-nhtsa-recalls-every-day-with-python-lm7</link>
      <guid>https://dev.to/martinpython/check-a-list-of-vins-for-new-nhtsa-recalls-every-day-with-python-lm7</guid>
      <description>&lt;p&gt;If you manage more than a handful of cars, finding out which of them have an open safety recall usually means typing VINs into the NHTSA website one at a time.&lt;/p&gt;

&lt;p&gt;That works for your own car. It does not work for a 40-van delivery fleet, a used-car lot, a rental desk or an insurance workflow where new vehicles arrive every week. And a recall is not a one-time fact: NHTSA publishes new campaigns all the time, so a car that was clean last month can have a new recall today.&lt;/p&gt;

&lt;p&gt;I built a small API for this, and in this post I show how to use it from Python: decode a list of VINs, attach the open NHTSA recalls, and then run it every day so you only get the recalls that are new since yesterday. All output below is real, from runs I made today (2026-11-12).&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;NHTSA has two free, official APIs: &lt;strong&gt;vPIC&lt;/strong&gt; (VIN decoding) and the &lt;strong&gt;Recalls API&lt;/strong&gt;. They are good, but using them directly for a fleet means:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;vPIC returns about 140 fields per VIN, most of them empty strings, and numbers come back as text.&lt;/li&gt;
&lt;li&gt;The Recalls API does not take a VIN. It takes make, model and model year, so you first decode every VIN, then normalize the make and model, then call recalls per vehicle.&lt;/li&gt;
&lt;li&gt;If NHTSA is slow or down, a naive script sees an empty answer and reports "0 recalls". For a safety check that is the worst possible failure.&lt;/li&gt;
&lt;li&gt;To get only &lt;em&gt;new&lt;/em&gt; recalls you need to store what you saw last time and diff it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of this is hard, but it is a day of glue code plus maintenance. The API below does it in one call.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the API does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Up to 50 VINs per call (and up to 20 vehicles by text like &lt;code&gt;2021 toyota camry&lt;/code&gt; when you have no VIN).&lt;/li&gt;
&lt;li&gt;Clean decode: make, model, year, trim, body, engine, drive type, transmission, plant, GVWR and safety equipment (ABS, ESC, TPMS, airbags, collision warning) where NHTSA has data.&lt;/li&gt;
&lt;li&gt;A local check digit test (&lt;code&gt;checkDigitValid&lt;/code&gt;), so typos are caught before any network call.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;recallCount&lt;/code&gt; and a &lt;code&gt;recalls&lt;/code&gt; list per vehicle: campaign number, component, summary, consequence, remedy, report date, park-it flags and the NHTSA link.&lt;/li&gt;
&lt;li&gt;If NHTSA does not answer after 3 retries, the row says &lt;code&gt;unavailable&lt;/code&gt;. It never says "no recalls" when it does not know.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;onlyNew&lt;/code&gt; + &lt;code&gt;monitorId&lt;/code&gt;: returns one row per recall that this monitor has not seen before.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It runs on Apify as &lt;a href="https://apify.com/martin10291/vehicle-recalls-vin-decoder" rel="noopener noreferrer"&gt;Vehicle Recalls &amp;amp; VIN Decoder API&lt;/a&gt;. You call it with your own Apify token.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get a token
&lt;/h2&gt;

&lt;p&gt;Create an Apify account (the free plan includes a small monthly usage credit), open &lt;strong&gt;Settings -&amp;gt; API &amp;amp; Integrations&lt;/strong&gt; and copy your personal API token. Put it in an environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"your_token_here"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: one call with curl
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s -X POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~vehicle-recalls-vin-decoder/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"vins": ["1HGCV1F35KA000001", "1FTFW1ET9DFC10312"], "vehicles": ["2021 toyota camry"], "includeRecalls": true}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;run-sync-get-dataset-items&lt;/code&gt; starts the run, waits for it and returns the rows. My run took 6.3 seconds for these 3 vehicles.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: the same in Python
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~vehicle-recalls-vin-decoder"&lt;/span&gt;
       &lt;span class="s2"&gt;"/run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;check_fleet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vins&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vehicles&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"vins"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;vins&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"vehicles"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;vehicles&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"includeRecalls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                      &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;check_fleet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"1HGCV1F35KA000001"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"1FTFW1ET9DFC10312"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"2021 toyota camry"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'{row["modelYear"]} {row["make"]} {row["model"]}'&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"vin"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"recalls:"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"recallCount"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;rc&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"recalls"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"   "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rc&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"campaignNumber"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rc&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"reportDate"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rc&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"component"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Real output from today:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ok 1HGCV1F35KA000001 2019 HONDA Accord recalls: 6
    26V332000 2026-05-22 AIR BAGS:SENSOR:OCCUPANT CLASSIFICATION:FRONT PASSENGER
    23V858000 2023-12-19 FUEL SYSTEM, GASOLINE:DELIVERY:FUEL PUMP
    23V158000 2023-03-09 SEAT BELTS:FRONT:BUCKLE ASSEMBLY
    21V215000 2021-03-25 FUEL SYSTEM, GASOLINE:DELIVERY:FUEL PUMP
    20V771000 2020-12-10 ELECTRICAL SYSTEM:BODY CONTROL MODULE:SOFTWARE
    20V314000 2020-05-28 FUEL SYSTEM, GASOLINE:DELIVERY:FUEL PUMP
ok 1FTFW1ET9DFC10312 2013 FORD F-150 recalls: 3
    19V433000 2019-06-10 POWER TRAIN:AUTOMATIC TRANSMISSION
    19V075000 2019-02-11 POWER TRAIN:AUTOMATIC TRANSMISSION:CONTROL MODULE (TCM/PCM/TECM)
    16V345000 2016-05-24 SERVICE BRAKES, HYDRAULIC:FLUID
ok None 2021 toyota camry recalls: 1
    23V865000 2023-12-20 AIR BAGS:SENSOR:OCCUPANT CLASSIFICATION
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And one decoded row, shortened (the &lt;code&gt;recalls&lt;/code&gt; list is cut to one entry):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"vehicle"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"vin"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1FTFW1ET9DFC10312"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"ok"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"checkDigitValid"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"make"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"FORD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"model"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"F-150"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"modelYear"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2013&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"bodyClass"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Pickup"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"vehicleType"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"TRUCK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"engine"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"cylinders"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"displacementL"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;3.5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"hp"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;365&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"fuelType"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Gasoline"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"configuration"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"V-Shaped"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"driveType"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"4WD/4-Wheel Drive/4x4"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"plantCity"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"DEARBORN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"plantState"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"MICHIGAN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"gvwr"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Class 2F: 7,001 - 8,000 lb (3,175 - 3,629 kg)"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"safety"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"tpms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Direct"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"airbagsFront"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1st Row (Driver and Passenger)"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"airbagsSide"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1st and 2nd Rows"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"recallCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"recalls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"campaignNumber"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"19V433000"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"component"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"POWER TRAIN:AUTOMATIC TRANSMISSION"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"consequence"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Unexpectedly downshifting into first gear may result in a loss of vehicle control, increasing the risk of a crash."&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"source"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"NHTSA vPIC + NHTSA Recalls API (US market vehicles)"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"checkedAt"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-12T14:31:42Z"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note the Toyota row: it came from the text &lt;code&gt;2021 toyota camry&lt;/code&gt;, so there is no VIN and no decoded specs, only the recalls for that make, model and year.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: only the new recalls, every day
&lt;/h2&gt;

&lt;p&gt;For a fleet you do not want the full list every morning. You want "what changed". Turn on &lt;code&gt;onlyNew&lt;/code&gt; and give your list a &lt;code&gt;monitorId&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;new_recalls&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vins&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;monitor_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"my-fleet"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"vins"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;vins&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"onlyNew"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;monitor_id&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                      &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;new_recalls&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"1HGCV1F35KA000001"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"1FTFW1ET9DFC10312"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"vin"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"campaignNumber"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"component"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"new recalls"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In monitor mode you get one row per recall (with the vehicle fields included), not one row per car. Here is what happened today with the same two VINs:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Run&lt;/th&gt;
&lt;th&gt;Rows&lt;/th&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1st run (new monitor)&lt;/td&gt;
&lt;td&gt;9 (all current recalls: 6 Honda + 3 Ford)&lt;/td&gt;
&lt;td&gt;4.2 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2nd run, a minute later&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;3.4 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The first run returns everything because the monitor has never seen these cars. From the second run on, you only get campaigns that NHTSA added since the last run. Each &lt;code&gt;monitorId&lt;/code&gt; keeps its own memory, so you can run one per customer or per depot.&lt;/p&gt;

&lt;p&gt;To run it daily, use either:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;cron on your own machine:&lt;/strong&gt; &lt;code&gt;0 7 * * * APIFY_TOKEN=... /usr/bin/python3 /home/you/fleet_recalls.py&lt;/code&gt;, and make the script send an e-mail or a Slack message only when &lt;code&gt;len(rows) &amp;gt; 0&lt;/code&gt;; or&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;an Apify schedule:&lt;/strong&gt; open the actor, save your input (with &lt;code&gt;onlyNew: true&lt;/code&gt; and your &lt;code&gt;monitorId&lt;/code&gt;) as a task, and add a daily schedule. You can attach a webhook to the task so the new rows go to your own endpoint.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Use it from an AI agent
&lt;/h2&gt;

&lt;p&gt;The actor also works through the Apify MCP server. Add &lt;code&gt;https://mcp.apify.com?tools=martin10291/vehicle-recalls-vin-decoder&lt;/code&gt; to Claude, Cursor or another MCP client and ask "Decode these VINs and list their open recalls".&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Recalls are matched by make, model and model year, not by the repair status of one VIN.&lt;/strong&gt; A listed recall means that model year is covered by the campaign. It does not tell you whether this particular car was already fixed. For that, the owner or a dealer has to check the individual VIN on NHTSA's own lookup page or with the manufacturer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;US market data.&lt;/strong&gt; vPIC is built for vehicles sold in the US and Canada. Many European or Asian market VINs decode only partly or not at all (&lt;code&gt;not_decoded&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Empty fields mean NHTSA has no data, not that the car lacks the feature.&lt;/li&gt;
&lt;li&gt;A make with several words (for example Land Rover) needs the object form in &lt;code&gt;vehicles&lt;/code&gt;: &lt;code&gt;{"year": 2020, "make": "land rover", "model": "defender"}&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;A make or model that NHTSA does not recognize returns 0 recalls, the same as a model with no recalls. Check &lt;code&gt;status&lt;/code&gt; and the decoded &lt;code&gt;make&lt;/code&gt; before you trust a 0.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;$1 per 1,000 result rows: one row per vehicle in normal mode, one row per new recall in monitor mode. A daily monitor run that finds nothing new returns 0 rows. The data itself is free and official; you pay for the decoding, the cleanup, the retries and the memory of what you already saw.&lt;/p&gt;

&lt;p&gt;If you try it on your own list and something decodes badly, open an issue on the actor page with the VIN pattern and I will look at it. What would make this more useful for your fleet: a CSV upload, or a direct e-mail when a new recall appears?&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/vehicle-recalls-vin-decoder?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/vehicle-recalls-vin-decoder&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>automation</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Get only the price changes from online stores every day with Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Thu, 12 Nov 2026 21:45:03 +0000</pubDate>
      <link>https://dev.to/martinpython/get-only-the-price-changes-from-online-stores-every-day-with-python-5595</link>
      <guid>https://dev.to/martinpython/get-only-the-price-changes-from-online-stores-every-day-with-python-5595</guid>
      <description>&lt;p&gt;If you watch competitor prices, you do not want 200 rows every morning. You want the 3 rows where something moved.&lt;/p&gt;

&lt;p&gt;Yesterday I wrote about &lt;a href="https://dev.to/martinpython/compare-one-products-price-across-online-stores-with-python-real-output-3dd5"&gt;comparing one product's price across online stores&lt;/a&gt;. At the end I showed a small manual diff and said a built-in change mode was next. It is live now, so this post shows how to use it: a daily run that returns only offers whose price or stock changed, with the old price, the new price and the change in percent. Real output from today (2026-11-11) included, plus the limits.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;A home-made price watch usually grows like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Scrape every product page on a schedule.&lt;/li&gt;
&lt;li&gt;Save all rows somewhere.&lt;/li&gt;
&lt;li&gt;Load yesterday's rows, match them by URL, compare price and stock.&lt;/li&gt;
&lt;li&gt;Decide what counts as a change (currency rounding, sale price, "out of stock").&lt;/li&gt;
&lt;li&gt;Store today's rows as the new baseline, without losing them when a run fails halfway.&lt;/li&gt;
&lt;li&gt;Pay for storing and processing hundreds of rows that did not change.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Steps 3 to 5 are the boring part, and they are where these scripts break: one failed run overwrites the baseline and the next day every row looks "new".&lt;/p&gt;

&lt;h2&gt;
  
  
  What the change mode does
&lt;/h2&gt;

&lt;p&gt;I built the Cheapest Price Finder: you give it product page URLs from any stores, it reads the real price from each page's structured data (JSON-LD offers, Shopify product data, meta price tags), converts everything to one currency with daily ECB rates and sorts the offers from cheapest to most expensive.&lt;/p&gt;

&lt;p&gt;Two inputs turn it into a price watch:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;onlyChanged: true&lt;/code&gt; saves only offers whose price or stock changed since the last run, plus pages it has never seen. Unchanged offers are not saved and not billed.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;monitorId&lt;/code&gt; (optional) names the watch, e.g. &lt;code&gt;aeropress-eu&lt;/code&gt;. Runs with the same &lt;code&gt;monitorId&lt;/code&gt; share the remembered prices even when you add or remove URLs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every returned row gets &lt;code&gt;changeType&lt;/code&gt; (&lt;code&gt;new&lt;/code&gt;, &lt;code&gt;priceDrop&lt;/code&gt;, &lt;code&gt;priceIncrease&lt;/code&gt;, &lt;code&gt;availability&lt;/code&gt;), &lt;code&gt;previousPrice&lt;/code&gt;, &lt;code&gt;priceChange&lt;/code&gt; and &lt;code&gt;priceChangePercent&lt;/code&gt;. These are in the shop's own currency (the &lt;code&gt;price&lt;/code&gt; field), so a moving exchange rate never looks like a price change. The state lives in your own Apify account, so your watch is yours alone.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get a token
&lt;/h2&gt;

&lt;p&gt;Create a free Apify account and copy your token from Console (Settings, API and Integrations). Put it in an environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;your_token_here
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: the first run with curl
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~cheapest-price-finder/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"productUrls": [
        "https://www.vervecoffee.com/products/aeropress-coffee-brewer",
        "https://www.intelligentsia.com/products/aeropress",
        "https://www.espressoparts.com/products/aeropress-coffee-maker",
        "https://www.stumptowncoffee.com/products/aeropress"],
      "convertTo": "EUR",
      "onlyChanged": true,
      "monitorId": "aeropress-eu"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first run has nothing to compare with, so every offer comes back as &lt;code&gt;new&lt;/code&gt;. That is your baseline.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: the same thing in Python
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;  &lt;span class="c1"&gt;# your own token&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~cheapest-price-finder/"&lt;/span&gt;
       &lt;span class="s2"&gt;"run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="no"&gt;URLS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.vervecoffee.com/products/aeropress-coffee-brewer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.intelligentsia.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.espressoparts.com/products/aeropress-coffee-maker"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.stumptowncoffee.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;price_changes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;watch&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"aeropress-eu"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;currency&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;180&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"productUrls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"convertTo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;currency&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"onlyChanged"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;watch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;price_changes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URLS&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"changeType"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"new"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"NEW   {o['store']:22} {o['priceConverted']:&amp;gt;7} EUR  {o['availability']}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{o['changeType']:13} {o['store']:22} "&lt;/span&gt;
              &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{o['previousPrice']} -&amp;gt; {o['price']} {o['currency']} "&lt;/span&gt;
              &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"({o['priceChangePercent']:+}%)  {o['availability']}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Real output from today
&lt;/h2&gt;

&lt;p&gt;I ran the same input twice, a few minutes apart, with &lt;code&gt;monitorId&lt;/code&gt; set.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Run 1&lt;/strong&gt; (9.5 s, 4 rows):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NEW   vervecoffee.com          35.63 EUR  InStock
NEW   intelligentsia.com       35.67 EUR  OutOfStock
NEW   espressoparts.com        40.08 EUR  InStock
NEW   stumptowncoffee.com      41.02 EUR  InStock
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One row as JSON (trimmed):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"rankByPrice"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"isCheapest"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"store"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"vervecoffee.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"price"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;39.95&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"currency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"USD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"priceConverted"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;35.63&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"convertedCurrency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"availability"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"InStock"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"changeType"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"new"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"previousPrice"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"priceChange"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"priceChangePercent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://www.vervecoffee.com/products/aeropress-coffee-brewer"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Run 2&lt;/strong&gt; (6.1 s, 0 rows). No price moved in those minutes, so the dataset is empty and the run was charged for 0 results. The run's &lt;code&gt;RUN_SUMMARY&lt;/code&gt; record tells you it still checked everything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"pagesChecked"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"offers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"failed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"lastError"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"monitor"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"devto-aeropress-1111"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"firstRun"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"onlyChanged"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"changes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"new"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"priceDrop"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"priceIncrease"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="nl"&gt;"availability"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"unchanged"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"rowsSaved"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I did not invent a price drop for this post. When one of these shops changes a price, the row comes back with &lt;code&gt;changeType: "priceDrop"&lt;/code&gt; or &lt;code&gt;"priceIncrease"&lt;/code&gt;, &lt;code&gt;previousPrice&lt;/code&gt;, &lt;code&gt;priceChange&lt;/code&gt; and &lt;code&gt;priceChangePercent&lt;/code&gt; filled. An "in stock" to "out of stock" switch comes back as &lt;code&gt;availability&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;An empty dataset on a quiet day is the point: your alert code runs only on real changes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: run it every day
&lt;/h2&gt;

&lt;p&gt;The simplest way is an Apify schedule:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open the &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;actor page&lt;/a&gt;, paste your URLs, tick "Only changed offers" and type a Monitor ID.&lt;/li&gt;
&lt;li&gt;Save it as a Task.&lt;/li&gt;
&lt;li&gt;Add a schedule, e.g. every day at 07:00.&lt;/li&gt;
&lt;li&gt;Optional: add a webhook on "run succeeded" that calls your own endpoint, or connect the dataset to Google Sheets, Slack or email with Apify integrations.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If you prefer your own cron, call &lt;code&gt;price_changes()&lt;/code&gt; from a script and send the rows wherever you like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;0 7 * * * /usr/bin/python3 /home/me/price_changes.py &amp;gt;&amp;gt; /home/me/prices.log 2&amp;gt;&amp;amp;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add or remove URLs at any time. With the same &lt;code&gt;monitorId&lt;/code&gt;, old URLs keep their remembered prices and new URLs come back as &lt;code&gt;new&lt;/code&gt; once.&lt;/p&gt;

&lt;h2&gt;
  
  
  The API.market option
&lt;/h2&gt;

&lt;p&gt;The same tool is on &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;API.market&lt;/a&gt; with a free plan, if you want one API key and simple HTTP:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s1"&gt;'https://prod.api.market/api/v1/martindata/price-comparison-api/m/cheapest-price-finder'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'x-api-market-key: YOUR_API_MARKET_KEY'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"productUrls": ["https://www.espressoparts.com/products/aeropress-coffee-maker"],
       "convertTo": "EUR"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One honest difference: on API.market the change mode is switched off. The remembered prices would live in my account, shared by every caller with the same URLs, and one customer's run would "use up" another customer's changes. So there you get the full sorted offers every time and keep the diff yourself (the 10-line diff from &lt;a href="https://dev.to/martinpython/compare-one-products-price-across-online-stores-with-python-real-output-3dd5"&gt;the first post&lt;/a&gt; works). For a built-in change feed, use the Apify actor with your own token.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Big marketplaces often block automated requests.&lt;/strong&gt; Amazon, Walmart, Target, Best Buy and Newegg frequently answer with a bot-check page. The tool does not bypass those; it reports them in &lt;code&gt;RUN_SUMMARY.failed&lt;/code&gt; and does not charge for them. Brand stores and independent shops (Shopify, WooCommerce, Magento, BigCommerce) work best.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The first run is never empty&lt;/strong&gt;: every page is &lt;code&gt;new&lt;/code&gt; once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No shipping costs or coupon codes.&lt;/strong&gt; It compares the listed product price.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It does not match products for you.&lt;/strong&gt; You choose which URLs are the same item; &lt;code&gt;gtin&lt;/code&gt;, &lt;code&gt;sku&lt;/code&gt; and &lt;code&gt;brand&lt;/code&gt; help you check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pages that build the price only with JavaScript&lt;/strong&gt; and publish no product data return "no price found".&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;On Apify it costs $3.00 per 1,000 saved results (lower with Apify's Bronze, Silver and Gold plans). With &lt;code&gt;onlyChanged&lt;/code&gt;, unchanged offers are not saved, so a quiet day costs nothing for results. My two runs above together cost under one tenth of a cent of platform usage. Apify's free plan includes a small monthly credit, enough to try this.&lt;/p&gt;

&lt;p&gt;On API.market there is a free plan to test, then the paid plans shown on the &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;listing&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because every price script I wrote ended up as the same "compare with yesterday" code, and it was the part that broke. Now the tool keeps yesterday for you.&lt;/p&gt;

&lt;p&gt;If you watch prices for a shop, a reseller business or a deal newsletter, try it on &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;Apify&lt;/a&gt; or &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;API.market&lt;/a&gt;. Which shop would you add to the watch first? If it returns "no price found", post the URL in the comments and I will look at it.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>ecommerce</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>Find companies hiring sales and engineering roles this week with Python (hiring signals API)</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Wed, 11 Nov 2026 21:45:03 +0000</pubDate>
      <link>https://dev.to/martinpython/find-companies-hiring-sales-and-engineering-roles-this-week-with-python-hiring-signals-api-1cgc</link>
      <guid>https://dev.to/martinpython/find-companies-hiring-sales-and-engineering-roles-this-week-with-python-hiring-signals-api-1cgc</guid>
      <description>&lt;p&gt;If you sell to tech companies, the best time to reach out is right after they start hiring for the team that would use your product, and most of us find out months later from a LinkedIn post.&lt;/p&gt;

&lt;p&gt;A company that opens 30 new sales roles in a week is about to buy sales tools. A company that adds 60 engineers is about to buy dev tools, cloud and hiring services. Job boards show this in public, but only as a snapshot of today. They do not tell you what is new, what closed, or how fast the count is moving.&lt;/p&gt;

&lt;p&gt;I built a small API that answers those questions from a daily archive of public job boards. In this post I show how to call it from Python and curl, the real output I got today (2026-11-07), and how to turn it into a daily "new sales jobs at companies I care about" feed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;The usual way to get hiring signals looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Find the job board of every company.&lt;/strong&gt; Greenhouse, Lever and Ashby each have a public JSON endpoint, but you need the board slug for each company first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Crawl hundreds of boards every day.&lt;/strong&gt; One company is easy. 800 companies every morning is a job queue, retries and rate limits.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Store every snapshot.&lt;/strong&gt; "New" and "closed" only exist if you kept yesterday's data. If you start today, you have no history until next week.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Diff the snapshots.&lt;/strong&gt; Same job, new ID? A posting that disappears for one day and comes back? You need rules for that.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Aggregate per company.&lt;/strong&gt; Open jobs then vs now, new jobs, closed jobs, remote share.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The data is free. The archive is the work. Paid intent-data tools do this for you, but they are priced for sales teams with a budget, not for a developer who wants a JSON list.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I built
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://apify.com/martin10291/company-hiring-signals-history" rel="noopener noreferrer"&gt;Company Hiring Signals API&lt;/a&gt; reads a daily archive of public Greenhouse, Lever and Ashby boards for &lt;strong&gt;813 tech and startup companies&lt;/strong&gt; (started 2026-11-03, one snapshot per day). Each run reads that archive, so it does not crawl anything on your time. Today's runs took 4 to 5 seconds end to end.&lt;/p&gt;

&lt;p&gt;It has five modes:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;mode&lt;/th&gt;
&lt;th&gt;what you get&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;top_hiring&lt;/code&gt; (default)&lt;/td&gt;
&lt;td&gt;companies ranked by new jobs, open jobs, change in open jobs or closed jobs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;companies&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;current numbers plus the daily history of the companies you list&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;new_jobs&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;postings first seen after the baseline, with keyword, location, department and remote filters&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;closed_jobs&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;postings that disappeared two days in a row, with &lt;code&gt;daysOpen&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;open_jobs&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;currently open postings, with filters&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;It only returns company data: no recruiter names, emails or other personal data.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get an Apify token
&lt;/h2&gt;

&lt;p&gt;The API runs on Apify. Create a free account, open &lt;strong&gt;Settings, API &amp;amp; Integrations&lt;/strong&gt; and copy your personal API token. Keep it in an environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"your-token-here"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: which companies are hiring fastest right now?
&lt;/h2&gt;

&lt;p&gt;One curl call. &lt;code&gt;run-sync-get-dataset-items&lt;/code&gt; starts the run, waits for it and returns the rows in the same HTTP response:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s -X POST \
  &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~company-hiring-signals-history/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"mode":"top_hiring","sortBy":"newJobs","limit":5}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is one real row from today's run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"company"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Samsara"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"companySlug"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"samsara"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"ats"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"greenhouse"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"day"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-07"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"comparedWith"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-03"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"openJobs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;363&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"openJobsThen"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;326&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"openJobsDelta"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;37&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"openJobsDeltaPct"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;11.3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"newJobs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;69&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"closedJobs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;32&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"remoteSharePct"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;85.4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"archiveFrom"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-03"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"archiveSnapshots"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And the top 4 by new jobs between 2026-11-03 and 2026-11-07:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;company&lt;/th&gt;
&lt;th&gt;open now&lt;/th&gt;
&lt;th&gt;open on 11-03&lt;/th&gt;
&lt;th&gt;new jobs&lt;/th&gt;
&lt;th&gt;closed jobs&lt;/th&gt;
&lt;th&gt;remote share&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Anduril Industries&lt;/td&gt;
&lt;td&gt;2,633&lt;/td&gt;
&lt;td&gt;2,564&lt;/td&gt;
&lt;td&gt;166&lt;/td&gt;
&lt;td&gt;97&lt;/td&gt;
&lt;td&gt;1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SpaceX&lt;/td&gt;
&lt;td&gt;2,930&lt;/td&gt;
&lt;td&gt;2,881&lt;/td&gt;
&lt;td&gt;124&lt;/td&gt;
&lt;td&gt;75&lt;/td&gt;
&lt;td&gt;0.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Samsara&lt;/td&gt;
&lt;td&gt;363&lt;/td&gt;
&lt;td&gt;326&lt;/td&gt;
&lt;td&gt;69&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;85.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Stripe&lt;/td&gt;
&lt;td&gt;783&lt;/td&gt;
&lt;td&gt;746&lt;/td&gt;
&lt;td&gt;62&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;14.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The raw count favours big companies, so &lt;code&gt;openJobsDeltaPct&lt;/code&gt; is often the more useful column: Samsara grew its open roles by 11.3% in four days, Anduril by 2.7%. Use &lt;code&gt;"sortBy":"openJobsDelta"&lt;/code&gt; and &lt;code&gt;minOpenJobs&lt;/code&gt; to rank by growth and hide tiny boards.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: new sales jobs as a lead list, in Python
&lt;/h2&gt;

&lt;p&gt;Here is the version I would actually use for outreach: new postings with sales titles, newest first. It needs only &lt;code&gt;requests&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~company-hiring-signals-history"&lt;/span&gt;
       &lt;span class="s2"&gt;"/run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"new_jobs"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"keywords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"account executive"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"sales"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"limit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                  &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="s2"&gt;"jobId"&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;          &lt;span class="c1"&gt;# skip the run summary row, if any&lt;/span&gt;
        &lt;span class="k"&gt;continue&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'{job["company"]:&amp;lt;22} {job["title"][:55]:&amp;lt;55} '&lt;/span&gt;
          &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'{job.get("location") or "-":&amp;lt;18} first seen {job["firstSeen"]}'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Five of the rows I got today (I ran it with &lt;code&gt;limit: 5&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;EliseAI                Sales Development Representative | Housing | Sales     New York City      first seen 2026-11-07
Weave                  Salesforce Development Manager &amp;amp; Systems Architect     Weave - Headquarters (Lehi, UT) first seen 2026-11-07
Aledade                Network Development Sales Lead                         Durham, NC         first seen 2026-11-07
Braze                  Account Executive, Enterprise                          Singapore          first seen 2026-11-07
Canarytechnologies     Associate Account Executive – Italy (Italian Speaking) Barcelona          first seen 2026-11-07
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Notice the second row. Keyword matching is a plain substring test today, so &lt;code&gt;sales&lt;/code&gt; also matches &lt;strong&gt;Sales&lt;/strong&gt;force. If you want only sales roles, filter by department too (&lt;code&gt;"department": ["Sales"]&lt;/code&gt;), or drop titles you do not want in your own code. Each row also has the job &lt;code&gt;url&lt;/code&gt;, &lt;code&gt;department&lt;/code&gt;, &lt;code&gt;remote&lt;/code&gt; and a short &lt;code&gt;snippet&lt;/code&gt; of the description, so you can tell your outreach tool why you are writing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: run it daily and get only what is new
&lt;/h2&gt;

&lt;p&gt;Add &lt;code&gt;onlyNew&lt;/code&gt; and a &lt;code&gt;monitorId&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"onlyNew"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"sales-roles"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first run is your baseline. After that, each run outputs only postings it has not reported to that &lt;code&gt;monitorId&lt;/code&gt; before, and you only pay for those rows. Use a different &lt;code&gt;monitorId&lt;/code&gt; per list (for example one for sales roles, one for your 50 target accounts with &lt;code&gt;"companies": ["stripe", "samsara", ...]&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;To schedule it you have two options:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apify Schedules&lt;/strong&gt;: open the actor, save your input as a task, then add a schedule such as &lt;code&gt;0 7 * * *&lt;/code&gt;. Connect it to a webhook, Slack, email, Google Sheets or Zapier/Make so the new rows land where you work. There is a ready example task on the actor page ("Which companies are hiring most").&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your own cron&lt;/strong&gt;: keep the Python script above and run it every morning:
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;0 7 * * * &lt;span class="nb"&gt;cd&lt;/span&gt; /home/me/hiring &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;xxx python3 new_sales_jobs.py &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; feed.log 2&lt;span class="o"&gt;&amp;gt;&amp;amp;&lt;/span&gt;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 5: closed roles and days open
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;closed_jobs&lt;/code&gt; mode returns postings that were missing from the board two days in a row, with &lt;code&gt;daysOpen&lt;/code&gt;. Recruiters use it to see which roles were filled or pulled, and how fast. Because the archive started on 2026-11-03, &lt;code&gt;daysOpen&lt;/code&gt; is still capped at a few days. It becomes more useful every day the archive runs, and that history is the part a new scraper cannot backfill.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Coverage is 813 companies&lt;/strong&gt;, mostly US tech and startups on Greenhouse, Lever and Ashby. It is not the whole job market. For a company that is not in the archive, my &lt;a href="https://apify.com/martin10291/ats-jobs-aggregator" rel="noopener noreferrer"&gt;ATS Jobs Aggregator&lt;/a&gt; reads any Greenhouse, Lever, Ashby or Workday board live.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The history is young.&lt;/strong&gt; Day one of the archive was 2026-11-03, so today &lt;code&gt;lookbackDays&lt;/code&gt; effectively means "since 11-03".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Departments are missing for Greenhouse boards today.&lt;/strong&gt; Greenhouse rows come back with &lt;code&gt;department: null&lt;/code&gt; and &lt;code&gt;topDepartments&lt;/code&gt; shows &lt;code&gt;(none)&lt;/code&gt;. Lever and Ashby rows have departments. Adding them is next on my list.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;New is relative to the baseline.&lt;/strong&gt; A company that reposts the same role with a new ID shows it as new plus closed. Small boards with heavy reposting can look busier than they are, so look at &lt;code&gt;openJobsDelta&lt;/code&gt; as well as &lt;code&gt;newJobs&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Company names come from the board slug&lt;/strong&gt; for some boards ("Canarytechnologies"), so match on &lt;code&gt;companySlug&lt;/code&gt; in your own data.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;Pay per result: $2.00 per 1,000 rows, no start fee. The 5-row runs above cost $0.01 each. In only-new mode you pay only for postings you have not seen. Apify's free plan includes a small monthly credit, which is enough to try it on your own watch list.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because hiring is one of the few buying signals that is both public and early, and I wanted it as a JSON list instead of a paid dashboard. If a company you care about is missing from the 813, open an issue on the actor page with its job board URL and I will add it to the daily archive.&lt;/p&gt;

&lt;p&gt;Next in this series: chaining the hiring list with domain and tech data to score accounts.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/company-hiring-signals-history?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/company-hiring-signals-history&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>sales</category>
      <category>jobs</category>
    </item>
    <item>
      <title>Compare one product's price across online stores with Python (real output)</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Tue, 10 Nov 2026 21:25:03 +0000</pubDate>
      <link>https://dev.to/martinpython/compare-one-products-price-across-online-stores-with-python-real-output-3dd5</link>
      <guid>https://dev.to/martinpython/compare-one-products-price-across-online-stores-with-python-real-output-3dd5</guid>
      <description>&lt;p&gt;If you sell a product, resell it, or just buy it for an office, you keep asking the same question: which store has it cheapest right now, in my currency?&lt;/p&gt;

&lt;p&gt;I wanted that answer as JSON from one call, so I built a small Cheapest Price Finder API. This post shows how to call it from curl and Python, the real output I got today (2026-11-10), how to run it every day and get only price changes, and what it does not do.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;The usual path for "compare a product across 4 shops" looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Write a scraper per store, because every shop puts the price somewhere else.&lt;/li&gt;
&lt;li&gt;Find out that some shops put it in JSON-LD, some only in Shopify's product JSON, some in a meta tag, and some render it with JavaScript.&lt;/li&gt;
&lt;li&gt;Handle variants, sale prices (&lt;code&gt;compareAtPrice&lt;/code&gt;) and "out of stock".&lt;/li&gt;
&lt;li&gt;Convert USD, EUR and GBP to one currency so the numbers are comparable.&lt;/li&gt;
&lt;li&gt;Sort, compute the difference to the cheapest, and save it somewhere.&lt;/li&gt;
&lt;li&gt;Fix it again when a store changes its theme.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of it is hard. It is just a day of glue code per store that you then maintain forever.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the API does
&lt;/h2&gt;

&lt;p&gt;You give it product page URLs from any stores (or a product name plus store domains). For each page it reads the real price from the page's structured data (JSON-LD offers, Shopify product data or meta price tags), converts it with the daily ECB reference rates, and returns every offer sorted from cheapest to most expensive.&lt;/p&gt;

&lt;p&gt;Each row has &lt;code&gt;rankByPrice&lt;/code&gt;, &lt;code&gt;isCheapest&lt;/code&gt;, &lt;code&gt;price&lt;/code&gt;, &lt;code&gt;currency&lt;/code&gt;, &lt;code&gt;priceConverted&lt;/code&gt;, &lt;code&gt;moreThanCheapest&lt;/code&gt; (amount and %), &lt;code&gt;availability&lt;/code&gt;, &lt;code&gt;brand&lt;/code&gt;, &lt;code&gt;gtin&lt;/code&gt;, &lt;code&gt;sku&lt;/code&gt;, &lt;code&gt;rating&lt;/code&gt; and &lt;code&gt;image&lt;/code&gt;. A &lt;code&gt;RUN_SUMMARY&lt;/code&gt; record lists the cheapest offer and every page it could not read, with the reason.&lt;/p&gt;

&lt;p&gt;It respects each store's robots.txt, sends an honest User-Agent and does not solve CAPTCHAs or bypass bot checks. A store that blocks automated requests is reported, not billed.&lt;/p&gt;

&lt;p&gt;You can call it two ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;API.market&lt;/strong&gt; (one API key, free plan to try): &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;Cheapest Price Finder on API.market&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Apify&lt;/strong&gt; (pay per result, good for schedules): &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;cheapest-price-finder on Apify&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Option 1: API.market with curl
&lt;/h2&gt;

&lt;p&gt;Subscribe to the free plan on the &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;API.market page&lt;/a&gt;, copy your key, then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s1"&gt;'https://prod.api.market/api/v1/martindata/price-comparison-api/m/cheapest-price-finder'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'x-api-market-key: YOUR_API_MARKET_KEY'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"productUrls": [
        "https://www.vervecoffee.com/products/aeropress-coffee-brewer",
        "https://www.intelligentsia.com/products/aeropress",
        "https://www.espressoparts.com/products/aeropress-coffee-maker",
        "https://www.stumptowncoffee.com/products/aeropress"],
      "convertTo": "EUR"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The response is JSON with the sorted offers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 2: Python through the Apify API
&lt;/h2&gt;

&lt;p&gt;Get a token in Apify Console (Settings, API and Integrations). Then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;  &lt;span class="c1"&gt;# your own token&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~cheapest-price-finder/"&lt;/span&gt;
       &lt;span class="s2"&gt;"run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;compare&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;currency&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                      &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"productUrls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"convertTo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;currency&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                      &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;180&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;urls&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s2"&gt;"https://www.espressoparts.com/products/aeropress-coffee-maker"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://vervecoffee.com/products/aeropress-coffee-brewer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://stumptowncoffee.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"https://intelligentsia.com/products/aeropress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;compare&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"#{o['rankByPrice']} {o['store']:22} {o['price']:&amp;gt;6} {o['currency']}"&lt;/span&gt;
          &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;" = {o['priceConverted']:&amp;gt;6} EUR  +{o['moreThanCheapestPercent']}%  {o['availability']}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I tested it with the AeroPress Original coffee maker, because four coffee shops sell the same item. This is what I got today. The run took 2.9 seconds on Apify:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;#1 vervecoffee.com         39.95 USD =  35.69 EUR  +0%     InStock
#2 intelligentsia.com      40    USD =  35.74 EUR  +0.1%   OutOfStock
#3 espressoparts.com       44.95 USD =  40.16 EUR  +12.5%  InStock
#4 stumptowncoffee.com     46    USD =  41.1  EUR  +15.2%  InStock
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the same product costs 15% more in one shop than in another, and the second cheapest is out of stock. That second fact is why I return &lt;code&gt;availability&lt;/code&gt; in every row: the cheapest offer is useless if you cannot buy it. Set &lt;code&gt;"onlyInStock": true&lt;/code&gt; to drop those rows.&lt;/p&gt;

&lt;p&gt;One full row (trimmed):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"rankByPrice"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"isCheapest"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"title"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AeroPress Coffee Maker - The Original 3-in-1 Coffee Brewer for Home, Travel &amp;amp; Adventure"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"store"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"espressoparts.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"price"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;44.95&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"currency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"USD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"priceConverted"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;40.16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"convertedCurrency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"moreThanCheapest"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;4.47&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"moreThanCheapestPercent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;12.5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"availability"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"InStock"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"brand"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AeroPress"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"gtin"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"085276000824"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"sku"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AEROPRESS"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"rating"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;4.89&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"reviewCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;9&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"priceSource"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"json-ld"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://www.espressoparts.com/products/aeropress-coffee-maker"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;gtin&lt;/code&gt; helps you check that two shops really sell the same item. In this run espressoparts.com and stumptowncoffee.com both show &lt;code&gt;085276000824&lt;/code&gt;. Not every shop publishes a GTIN, so treat it as a check, not a guarantee.&lt;/p&gt;

&lt;h2&gt;
  
  
  Searching by product name (best effort)
&lt;/h2&gt;

&lt;p&gt;If you do not have the URLs yet, you can pass a name and store domains:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;180&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"searchQuery"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AeroPress coffee maker"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"stores"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"aeropress.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"walmart.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ebay.com"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"resultsPerStore"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"convertTo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I will be honest about this mode: in my test today it found 0 offers. Walmart answered with a bot-check page (the tool does not solve those, it reports them), the search returned the AeroPress home and collection pages instead of a product page, and the search engine rate-limited the run. The &lt;code&gt;RUN_SUMMARY&lt;/code&gt; said exactly that and suggested adding product URLs. Product URLs are the reliable input; use search only to discover candidates.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run it every day and keep only price changes
&lt;/h2&gt;

&lt;p&gt;Price checks are useful only when they repeat. Save the input above as a Task in Apify Console and add a daily schedule. Then compare each run with the previous one. The actor has no built-in price-change mode yet, so here is the small diff I use:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt;

&lt;span class="n"&gt;state&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"last_prices.json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;old&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exists&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;offers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;compare&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;urls&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;new&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"priceConverted"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;offers&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;offers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;old&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"priceConverted"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{o['store']}: {before} -&amp;gt; {o['priceConverted']} EUR"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run it from cron on any small server, or from an Apify schedule with a webhook to your own endpoint, and you get a message only when a price moves. A built-in mode for this (only changed prices, per watch list) is the next thing I am adding.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Big marketplaces often block automated requests.&lt;/strong&gt; Amazon, Walmart, Target, Best Buy and Newegg frequently answer with a bot-check page. You get a clear reason in &lt;code&gt;RUN_SUMMARY.failed&lt;/code&gt; and you are not charged. It works best with brand stores and independent shops (Shopify, WooCommerce, Magento, BigCommerce).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pages that build the price only with JavaScript and publish no product data&lt;/strong&gt; return "no price found".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No shipping costs or coupon codes.&lt;/strong&gt; It compares the listed product price.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Currency conversion uses daily ECB reference rates&lt;/strong&gt;, not your card's rate.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It does not match products for you.&lt;/strong&gt; You decide which URLs are the same item; &lt;code&gt;gtin&lt;/code&gt;, &lt;code&gt;sku&lt;/code&gt; and &lt;code&gt;brand&lt;/code&gt; help.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apify:&lt;/strong&gt; $3.00 per 1,000 results, and only offers with a price count as results (failed pages are free). The 4-store run above cost about one cent. Apify's free plan includes a small monthly credit, enough to try it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;API.market:&lt;/strong&gt; a free plan to test, then the paid plans shown on the &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;listing&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because I kept writing the same "find the price on this page" code for every new shop. If you run a small reseller business, a deal newsletter, procurement for an office, or a repricing script, try the free plan on &lt;a href="https://api.market/store/martindata/price-comparison-api" rel="noopener noreferrer"&gt;API.market&lt;/a&gt; or the &lt;a href="https://apify.com/martin10291/cheapest-price-finder" rel="noopener noreferrer"&gt;Apify actor&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Which stores do you need to compare? If one of them returns "no price found", post the URL in the comments and I will look at it.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/cheapest-price-finder?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/cheapest-price-finder&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>ecommerce</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>Check your iOS app's App Store rank in 155 countries with Python (free)</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Mon, 09 Nov 2026 21:05:03 +0000</pubDate>
      <link>https://dev.to/martinpython/check-your-ios-apps-app-store-rank-in-155-countries-with-python-free-5cbg</link>
      <guid>https://dev.to/martinpython/check-your-ios-apps-app-store-rank-in-155-countries-with-python-free-5cbg</guid>
      <description>&lt;p&gt;If you ship an iOS app, App Store Connect tells you downloads and revenue, but not where your app sits in the top charts of each country.&lt;/p&gt;

&lt;p&gt;That number matters more than it looks. A jump from #80 to #30 in the Travel chart in Germany usually means a feature, a review, or a competitor slipping, and you want to know the day it happens, not a month later.&lt;/p&gt;

&lt;p&gt;I built a small free tool for this, and in this post I show both ways: the do-it-yourself script against Apple's own feed, and the shortcut.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;Apple publishes the top charts as a public RSS/JSON feed, one URL per country, per chart (free, paid, grossing) and per genre. That is the honest source, and you can use it yourself.&lt;/p&gt;

&lt;p&gt;The catch is the multiplication:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;155 App Store countries&lt;/li&gt;
&lt;li&gt;3 charts (free, paid, grossing)&lt;/li&gt;
&lt;li&gt;the overall chart plus your genre chart&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That is several hundred requests for one check. At a polite 2 requests per second it takes minutes, and if you go faster Apple may answer with HTTP 429. Commercial rank trackers solve this, but they are monthly subscriptions, often more than a small indie app earns.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 1: check it yourself with Apple's feed (Python)
&lt;/h2&gt;

&lt;p&gt;This script checks one app in 8 countries, in its genre chart (Travel here), free and grossing. It needs only &lt;code&gt;requests&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;APP_ID&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"1358823008"&lt;/span&gt;          &lt;span class="c1"&gt;# your app's numeric App Store id (from its URL: .../id1358823008)&lt;/span&gt;
&lt;span class="no"&gt;GENRE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;6003&lt;/span&gt;                   &lt;span class="c1"&gt;# Travel; 0 = all apps&lt;/span&gt;
&lt;span class="no"&gt;COUNTRIES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"us"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"gb"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"de"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"fr"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"jp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"au"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ca"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"in"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="no"&gt;CHARTS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"free"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"topfreeapplications"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"paid"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"toppaidapplications"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"grossing"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"topgrossingapplications"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="no"&gt;UA&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"User-Agent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"my-rank-check/1.0 (you@example.com)"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;rank&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cc&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;chart&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;genre&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;g&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"/genre={genre}"&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;genre&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;
    &lt;span class="n"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"https://itunes.apple.com/{cc}/rss/{CHARTS[chart]}/limit=100{g}/json"&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="no"&gt;UA&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;entries&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"feed"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"entry"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;pos&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"attributes"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"im:id"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="no"&gt;APP_ID&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;pos&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;cc&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;COUNTRIES&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;chart&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"free"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"grossing"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{cc.upper()} {chart:8} {rank(cc, chart, GENRE) or '-'}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;          &lt;span class="c1"&gt;# be polite: max 2 requests per second&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I ran it today (2026-11-09) for Flighty, a flight tracker I use as an example because it is well known. Real output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;US free     75
US grossing 2
GB free     80
GB grossing 2
DE free     -
DE grossing 2
FR free     -
FR grossing 4
JP free     -
JP grossing 6
AU free     88
AU grossing 2
CA free     -
CA grossing 2
IN free     -
IN grossing 2
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;-&lt;/code&gt; means the app is not in the top 100 of that chart. 16 requests took 23.5 seconds on my server. Scale that to 155 countries and all charts and you see the problem. Genre ids you will need: 6000 Business, 6002 Utilities, 6003 Travel, 6005 Social Networking, 6007 Productivity, 6013 Health &amp;amp; Fitness, 6014 Games, 6015 Finance, 6017 Education, 6024 Shopping.&lt;/p&gt;

&lt;p&gt;Tips if you keep the DIY route:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Send an honest User-Agent with a contact address.&lt;/li&gt;
&lt;li&gt;Keep it at 2 requests per second or less and back off on HTTP 429.&lt;/li&gt;
&lt;li&gt;Store every result with the date. The feed only shows today. Yesterday's chart is gone forever if you did not save it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Option 2: the free rank check I built
&lt;/h2&gt;

&lt;p&gt;The last tip is why I built this. Since 2026-11-07 a job on my server saves the official Apple charts every day:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;all 155 countries: top free, top paid and top grossing (overall)&lt;/li&gt;
&lt;li&gt;20 bigger countries (US, GB, DE, FR, JP, KR, CA, AU, BR, IN, IT, ES, MX, NL, SE, TR, ID, SA, PL, BG): top free and top grossing for 24 genres&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That is about 1,400 charts and 46,781 different apps so far. You can search any app by name or id here, free, no signup:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://2-29-135-242.sslip.io/apprank?src=devto" rel="noopener noreferrer"&gt;Check your app's rank in every country&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;There is also a plain JSON endpoint, so you can use it from a script or a cron job:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="s2"&gt;"https://2-29-135-242.sslip.io/apprank/q?q=1358823008"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Real response today (trimmed to the first 2 of 25 chart positions):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"q"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1358823008"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"day"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-09"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"apps"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1358823008&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Flighty – Live Flight Tracker"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"artist"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Flighty LLC"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"genre"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Travel"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"n"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;25&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"best"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"cc"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"IN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"chart"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"grossing"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"genre"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Travel"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"rank"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"ranks"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"cc"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"IN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"chart"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"grossing"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"genre"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Travel"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"rank"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"cc"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AU"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"chart"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"grossing"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"genre"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Travel"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"rank"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
      &lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One call returns every chart where the app appears, across all stored countries. The numbers match the DIY script above (US Travel grossing #2), because both read the same Apple feed. The difference is that the 1,400 requests already happened at night.&lt;/p&gt;

&lt;p&gt;The same in Python, with a small table:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://2-29-135-242.sslip.io/apprank/q"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"q"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1358823008"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"apps"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"- in"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"n"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"charts on"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"day"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"ranks"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"rank"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'  {x["cc"]}  {x["chart"]:8}  {x["genre"]:12}  #{x["rank"]}'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can also search by name (&lt;code&gt;q=flighty&lt;/code&gt;); it returns up to 5 matching apps.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run it daily
&lt;/h2&gt;

&lt;p&gt;To get a history of your own app, call the endpoint once a day and append to a CSV:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;APP&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"1358823008"&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://2-29-135-242.sslip.io/apprank/q"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"q"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;APP&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ranks.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"a"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;newline&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"apps"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"ranks"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writerow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"day"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"cc"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"chart"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"genre"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"rank"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# crontab -e: every day at 06:00 (the charts are saved around 01:30 UTC)&lt;/span&gt;
0 6 * * * &lt;span class="nb"&gt;cd&lt;/span&gt; /home/me/ranks &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; python3 daily.py
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;It only knows the top 100 of each chart. If your app is #350 in its genre, it will show nothing. That is a limit of Apple's public feed, not something I can fix.&lt;/li&gt;
&lt;li&gt;Genre charts are saved for 20 countries only. The other 135 countries have the overall free, paid and grossing charts.&lt;/li&gt;
&lt;li&gt;History starts on 2026-11-07, so today there are only a few days.&lt;/li&gt;
&lt;li&gt;iPhone App Store only, no Google Play, no keyword rankings.&lt;/li&gt;
&lt;li&gt;The endpoint is rate-limited (30 searches per 10 minutes per IP). It is fine for your own apps, not for bulk scraping.&lt;/li&gt;
&lt;li&gt;It is a one-person project on one server. If it is down, the DIY script above still works.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price, and a question for you
&lt;/h2&gt;

&lt;p&gt;The rank check and the JSON endpoint are free.&lt;/p&gt;

&lt;p&gt;I am deciding whether to build a proper tracker on top of the archive: daily email when your app enters, leaves or moves in any country, plus charts of the history. It would be $9 per month, and the first 3 months free for people on the waitlist. I only build it if enough developers actually want it, so there is a waitlist box on the same page: &lt;a href="https://2-29-135-242.sslip.io/apprank?src=devto" rel="noopener noreferrer"&gt;apprank&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you track ranks today, what do you use and what is missing? I read every comment.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ios</category>
      <category>python</category>
      <category>appstore</category>
      <category>indiedev</category>
    </item>
    <item>
      <title>Airport distance, flight time, CO2 and live METAR from IATA codes in Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Sun, 08 Nov 2026 20:45:04 +0000</pubDate>
      <link>https://dev.to/martinpython/airport-distance-flight-time-co2-and-live-metar-from-iata-codes-in-python-11b3</link>
      <guid>https://dev.to/martinpython/airport-distance-flight-time-co2-and-live-metar-from-iata-codes-in-python-11b3</guid>
      <description>&lt;p&gt;If your app has a "from" and a "to" airport, sooner or later you need the distance, a rough flight time, the time zone gap and maybe the CO2 per passenger, and you need it from a 3-letter code a user typed.&lt;/p&gt;

&lt;p&gt;I needed exactly that for a small travel side project, so I built an Airport Database API. This post shows how to call it from Python and curl, with the real output I got today (2026-11-08), how to schedule a daily weather check, and what it does not do.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;The usual path looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Download the OurAirports CSV (about 80,000 rows, most of them heliports and closed strips).&lt;/li&gt;
&lt;li&gt;Filter it, map IATA to ICAO, fix missing codes.&lt;/li&gt;
&lt;li&gt;Write a haversine function, then argue with yourself about what "flight time" means.&lt;/li&gt;
&lt;li&gt;Add a time zone library for the coordinates.&lt;/li&gt;
&lt;li&gt;For weather, learn the NOAA Aviation Weather API and decode METAR strings.&lt;/li&gt;
&lt;li&gt;Re-download the CSV every few weeks because airports change.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of this is hard, but it is a day of glue code you then have to maintain. I wanted one call that takes &lt;code&gt;["SOF-VAR", "LHR-JFK"]&lt;/code&gt; and returns everything as JSON.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the API does
&lt;/h2&gt;

&lt;p&gt;Five modes, one endpoint:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Mode&lt;/th&gt;
&lt;th&gt;Input&lt;/th&gt;
&lt;th&gt;Output&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;distance&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;routes like &lt;code&gt;JFK-LHR&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;km, miles, nautical miles, bearing, estimated flight time, time zone difference, CO2 per passenger (DEFRA 2025 factors)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;lookup&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;IATA or ICAO codes&lt;/td&gt;
&lt;td&gt;name, city, country, coordinates, elevation, time zone, runways&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;search&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;text, country, type&lt;/td&gt;
&lt;td&gt;matching airports&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;nearby&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;latitude, longitude, radius&lt;/td&gt;
&lt;td&gt;airports sorted by distance&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;weather&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;ICAO codes&lt;/td&gt;
&lt;td&gt;latest METAR (raw and decoded), optional TAF&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Data comes from OurAirports (public domain), the official NOAA Aviation Weather Center API and the UK DEFRA 2025 conversion factors. No scraping of websites, no API key for the sources.&lt;/p&gt;

&lt;p&gt;You can call it two ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;API.market&lt;/strong&gt; (simple API key, free plan to try): &lt;a href="https://api.market/store/martindata/airportsflightdistan-api" rel="noopener noreferrer"&gt;Airport Database API on API.market&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Apify&lt;/strong&gt; (pay per result, good for schedules): &lt;a href="https://apify.com/martin10291/airports-flight-distance" rel="noopener noreferrer"&gt;airports-flight-distance on Apify&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Option 1: API.market with curl
&lt;/h2&gt;

&lt;p&gt;Subscribe to the free plan on the &lt;a href="https://api.market/store/martindata/airportsflightdistan-api" rel="noopener noreferrer"&gt;API.market page&lt;/a&gt;, copy your key, then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s1"&gt;'https://prod.api.market/api/v1/martindata/airportsflightdistan-api/m/airports-flight-distance'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'x-api-market-key: YOUR_API_MARKET_KEY'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"mode": "distance", "routes": ["SOF-VAR", "LHR-JFK", "FRA-SIN"], "maxItems": 3}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The response is &lt;code&gt;{"count": N, "items": [...]}&lt;/code&gt;. On API.market one request returns up to 100 items.&lt;/p&gt;

&lt;h2&gt;
  
  
  Option 2: Python through the Apify API
&lt;/h2&gt;

&lt;p&gt;Get a token in Apify Console (Settings, Integrations). Then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;  &lt;span class="c1"&gt;# your own token&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~airports-flight-distance/"&lt;/span&gt;
       &lt;span class="s2"&gt;"run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;airports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;120&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;routes&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;airports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"distance"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"routes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"SOF-VAR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"LHR-JFK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"FRA-SIN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;routes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{r['route']:8} {r['greatCircleKm']:&amp;gt;8} km  {r['estimatedFlightTime']:&amp;gt;7}  "&lt;/span&gt;
          &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"tz {r['timezoneDifferenceHours']:+}h  CO2 {r['co2']['kgPerPassenger']} kg"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is what I got today. The run took 5.3 seconds:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SOF-VAR     363.5 km   0h 56m  tz +0h  CO2 51.8 kg
LHR-JFK    5539.8 km   7h 10m  tz -5h  CO2 846.6 kg
FRA-SIN   10280.2 km  12h 53m  tz +7h  CO2 1465.2 kg
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One full item looks like this (trimmed):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"route"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"LHR-JFK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"from"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"iata"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"LHR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"icao"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"EGLL"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"London Heathrow Airport"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"countryCode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"GB"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"to"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"iata"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"JFK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"icao"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"KJFK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"John F. Kennedy International Airport"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"countryCode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"US"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"greatCircleKm"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;5539.8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"miles"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;3442.3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"nauticalMiles"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;2991.3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"initialBearingCompass"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"WNW"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"estimatedFlightTime"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"7h 10m"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"estimateAssumptions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Estimate only: block time = distance / 830 km/h + 30 min taxi/climb/descent overhead. Not a schedule."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"timezoneDifferenceHours"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;-5.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"co2"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"defraCategory"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"long-haul"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"kgPerPassenger"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;846.6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"kgPerPassengerWithoutRF"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;501.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"includesRadiativeForcing"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The flight time is an estimate and says so in every row. I would rather ship an honest rough number than a fake schedule.&lt;/p&gt;

&lt;h2&gt;
  
  
  Nearby airports
&lt;/h2&gt;

&lt;p&gt;I live in Varna, Bulgaria, so this was my first test: which airports with scheduled flights are within 150 km of the city center?&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;near&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;airports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"nearby"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"latitude"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;43.2141&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"longitude"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;27.9147&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="s2"&gt;"radiusKm"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;150&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"maxItems"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;near&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"iata"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"distanceKm"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"km"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"countryCode"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Real output (2.8 seconds):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;VAR Varna Airport 7.5 km BG
BOJ Burgas Airport 78.7 km BG
CND Mihail Kogălniceanu International Airport 135.7 km RO
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Only 3 rows, because by default nearby mode keeps airports with scheduled service. Set &lt;code&gt;"scheduledServiceOnly": false&lt;/code&gt; if you also want small airfields.&lt;/p&gt;

&lt;h2&gt;
  
  
  Live METAR weather
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;wx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;airports&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"weather"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"codes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"LBWN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"EGLL"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;wx&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"icao"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"flightCategory"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{w['temperatureC']}C"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"wind {w['windDirectionDeg']}/{w['windSpeedKt']}kt"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"|"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"rawMetar"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Today at 14:00-14:20 UTC:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;EGLL MVFR 10C wind 310/4kt | METAR EGLL 081420Z COR AUTO 31004KT 250V340 7000 OVC012 10/07 Q1020 NOSIG
LBWN VFR 8C wind 130/7kt | METAR LBWN 081400Z AUTO 13007KT 9999 NCD 08/M01 Q1030 NOSIG
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each row has the raw METAR plus decoded fields (wind, gusts, visibility, temperature, dew point, pressure in hPa and inHg, clouds, flight category). Add &lt;code&gt;"includeTaf": true&lt;/code&gt; for the forecast.&lt;/p&gt;

&lt;h2&gt;
  
  
  Schedule it: only changed weather
&lt;/h2&gt;

&lt;p&gt;For a daily or hourly check you usually want only what changed. Turn on only-new mode and give the watch list a name:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"weather"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"codes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"LBWN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"LBBG"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"LBSF"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"onlyNew"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"bulgaria-airports"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Save that as a Task in Apify Console and add a schedule (for example every 30 minutes). Each run returns, and charges for, only the METARs that changed since the last run for that &lt;code&gt;monitorId&lt;/code&gt;. Connect the dataset to Slack, email, Google Sheets or a webhook from the same screen, and you have a small weather alert feed with no server of your own.&lt;/p&gt;

&lt;p&gt;In the other modes only-new returns airports or routes you have not seen before under that &lt;code&gt;monitorId&lt;/code&gt;, which is useful if you sync a growing route list into your database.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No live flights, timetables, delays or fares.&lt;/strong&gt; It is airport data, distances and weather. If you need flight status, this is the wrong tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Flight time is an estimate&lt;/strong&gt; (distance / 830 km/h + 30 min). Real block times vary with wind, routing and aircraft.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CO2 is a per-passenger estimate for a direct flight&lt;/strong&gt; using DEFRA 2025 factors, not an airline- or aircraft-specific number.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Small airfields often have no IATA code.&lt;/strong&gt; Use the ICAO code or search mode. Codes not found are listed in the run summary and are not charged.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Airport data is refreshed every 7 days&lt;/strong&gt; from OurAirports, which is community maintained.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apify:&lt;/strong&gt; $1.00 per 1,000 results (one route, airport or weather report = one result). The three test runs above returned 8 results, less than one cent. Apify's free plan has a small monthly credit, enough to try it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;API.market:&lt;/strong&gt; a free plan to test, then the paid plans shown on the &lt;a href="https://api.market/store/martindata/airportsflightdistan-api" rel="noopener noreferrer"&gt;listing&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because I kept rewriting the same haversine plus CSV plus METAR code. If you run a travel site, a chatbot that answers "how far is X from Y", or a small ops dashboard, try the free plan on &lt;a href="https://api.market/store/martindata/airportsflightdistan-api" rel="noopener noreferrer"&gt;API.market&lt;/a&gt; or the &lt;a href="https://apify.com/martin10291/airports-flight-distance" rel="noopener noreferrer"&gt;Apify actor&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What field is missing for your use case? Runway details, more emission methods, something else? Tell me in the comments and I will look at it.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/airports-flight-distance?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/airports-flight-distance&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>travel</category>
      <category>aviation</category>
    </item>
    <item>
      <title>Check domain expiry, DNS and SSL for a list of domains with one API call in Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Sat, 07 Nov 2026 20:25:04 +0000</pubDate>
      <link>https://dev.to/martinpython/check-domain-expiry-dns-and-ssl-for-a-list-of-domains-with-one-api-call-in-python-2npp</link>
      <guid>https://dev.to/martinpython/check-domain-expiry-dns-and-ssl-for-a-list-of-domains-with-one-api-call-in-python-2npp</guid>
      <description>&lt;p&gt;A domain you depend on expires or its SSL certificate runs out, and you learn about it from an angry customer instead of from a script.&lt;/p&gt;

&lt;p&gt;If you look after more than a handful of domains (your own, your clients', your vendors'), the facts you need live in four different places: registration data in WHOIS/RDAP, records in DNS, the certificate in the TLS handshake, and redirects in HTTP. Checking them by hand once a quarter is how expiry surprises happen.&lt;/p&gt;

&lt;p&gt;I built a small tool that collects all four in one call and returns one JSON row per domain. In this post I show how to use it from Python and curl, with the real output I got today, and how to run it daily so you only see domains that changed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;The usual path looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;WHOIS over port 43.&lt;/strong&gt; Every registry formats its text differently, many redact fields, and parsing "Registry Expiry Date:" vs "Expiration Time:" vs "paid-till:" with regexes is a hobby nobody wants.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A DNS library&lt;/strong&gt; for MX, TXT, SPF and DMARC, plus some code to find the SPF record among 30 TXT records.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An &lt;code&gt;ssl&lt;/code&gt; socket&lt;/strong&gt; to read &lt;code&gt;notAfter&lt;/code&gt; and compute days left.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Requests with redirects&lt;/strong&gt; to see where &lt;code&gt;http://domain&lt;/code&gt; ends up.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;State&lt;/strong&gt;, so tomorrow's run can tell you what changed instead of dumping everything again.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of it is hard, but it is four libraries, a lot of edge cases and a state file. The modern fix for step 1 is RDAP: the official JSON protocol that registries run today. The IANA bootstrap file (&lt;code&gt;data.iana.org/rdap/dns.json&lt;/code&gt;) tells you which RDAP server serves which TLD.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I built
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://apify.com/martin10291/domain-whois-dns-lookup" rel="noopener noreferrer"&gt;Domain WHOIS Lookup API&lt;/a&gt; is an Apify actor that does steps 1-5 for a list of domains:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Registration&lt;/strong&gt; from the official RDAP servers: registrar, created/updated/expiry dates, &lt;code&gt;domainAgeDays&lt;/code&gt;, &lt;code&gt;daysToExpiry&lt;/code&gt;, EPP status codes, nameservers, DNSSEC.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;DNS&lt;/strong&gt;: A, AAAA, MX, NS, TXT, CAA, plus parsed &lt;code&gt;SPF&lt;/code&gt; and &lt;code&gt;DMARC&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;SSL&lt;/strong&gt;: issuer, &lt;code&gt;notBefore&lt;/code&gt;, &lt;code&gt;notAfter&lt;/code&gt;, &lt;code&gt;daysLeft&lt;/code&gt;, TLS version, SANs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Redirects&lt;/strong&gt;: the chain from &lt;code&gt;http://domain&lt;/code&gt; and the final URL with status.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Only-new mode&lt;/strong&gt;: with a &lt;code&gt;monitorId&lt;/code&gt;, the next run outputs only domains whose registration, DNS or certificate changed, with a &lt;code&gt;changes&lt;/code&gt; list.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;No API key for any registry, no login, no proxies. It never outputs registrant names, emails or phones: you get the registrar (a company) and technical facts only.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get an Apify token
&lt;/h2&gt;

&lt;p&gt;Create a free Apify account and copy your API token from &lt;strong&gt;Settings -&amp;gt; API &amp;amp; Integrations&lt;/strong&gt;. Keep it in an environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"your-token-here"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: one call with curl
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s -X POST \
  &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~domain-whois-dns-lookup/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"domains":["python.org","dev.to","stripe.com","example.com"]}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;run-sync-get-dataset-items&lt;/code&gt; starts the run, waits for it and returns the rows in the same HTTP response. For these 4 domains the whole run took about 3.5 seconds today, and the lookups themselves 0.7 seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Real output (run on 2026-11-06)
&lt;/h2&gt;

&lt;p&gt;Here is the &lt;code&gt;example.com&lt;/code&gt; row, trimmed only where noted:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"example.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"registrar"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"RESERVED-Internet Assigned Numbers Authority"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"createdDate"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1995-08-14T04:00:00Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"expiryDate"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2027-08-13T04:00:00Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"domainAgeDays"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;11407&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"daysToExpiry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;279&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"client delete prohibited"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"client transfer prohibited"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"client update prohibited"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"nameservers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"elliott.ns.cloudflare.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"hera.ns.cloudflare.com"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"dnssec"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"rdapSource"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://rdap.verisign.com/com/v1/domain/example.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"dns"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"MX"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"0 ."&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"SPF"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"v=spf1 -all"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"DMARC"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"v=DMARC1;p=reject;sp=reject;adkim=s;aspf=s"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"A"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"104.20.23.154"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"172.66.147.243"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"ssl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"valid"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"issuer"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"SSL Corporation"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"notAfter"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-12-25T22:56:35Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"daysLeft"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;49&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"tlsVersion"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"TLSv1.3"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"redirect"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"finalUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"http://example.com/"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"finalStatus"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"changes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"new"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And the short version of all four:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;domain&lt;/th&gt;
&lt;th&gt;registrar&lt;/th&gt;
&lt;th&gt;days to expiry&lt;/th&gt;
&lt;th&gt;SSL days left&lt;/th&gt;
&lt;th&gt;SSL issuer&lt;/th&gt;
&lt;th&gt;DMARC policy&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;example.com&lt;/td&gt;
&lt;td&gt;IANA (reserved)&lt;/td&gt;
&lt;td&gt;279&lt;/td&gt;
&lt;td&gt;49&lt;/td&gt;
&lt;td&gt;SSL Corporation&lt;/td&gt;
&lt;td&gt;reject&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;stripe.com&lt;/td&gt;
&lt;td&gt;SafeNames Ltd.&lt;/td&gt;
&lt;td&gt;308&lt;/td&gt;
&lt;td&gt;184&lt;/td&gt;
&lt;td&gt;DigiCert, Inc.&lt;/td&gt;
&lt;td&gt;reject&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;python.org&lt;/td&gt;
&lt;td&gt;Gandi SAS&lt;/td&gt;
&lt;td&gt;2333&lt;/td&gt;
&lt;td&gt;99&lt;/td&gt;
&lt;td&gt;GlobalSign nv-sa&lt;/td&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;dev.to&lt;/td&gt;
&lt;td&gt;Tonic&lt;/td&gt;
&lt;td&gt;149&lt;/td&gt;
&lt;td&gt;190&lt;/td&gt;
&lt;td&gt;GlobalSign nv-sa&lt;/td&gt;
&lt;td&gt;reject&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A few things you get for free from one row: python.org is registered until 2033, its DMARC policy is &lt;code&gt;p=none&lt;/code&gt; (monitoring only), and its SPF includes Google, Mailgun and a status-page provider. stripe.com has 30 TXT records and no SPF record at the apex (&lt;code&gt;SPF: []&lt;/code&gt;), but a strict DMARC &lt;code&gt;p=reject&lt;/code&gt;. The tool keeps what DNS says and does not guess.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: a Python script for your own list
&lt;/h2&gt;

&lt;p&gt;Put your domains in &lt;code&gt;domains.txt&lt;/code&gt;, one per line. This script calls the actor, writes a CSV and prints anything that needs attention in the next 30 days:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~domain-whois-dns-lookup/"&lt;/span&gt;
       &lt;span class="s2"&gt;"run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="no"&gt;WARN_DAYS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;

&lt;span class="n"&gt;domains&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"domains.txt"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"domains"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;domains&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"includeDns"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"includeSsl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"includeRedirects"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  &lt;span class="c1"&gt;# faster if you only care about expiry&lt;/span&gt;
    &lt;span class="s2"&gt;"maxConcurrency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"domains.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;newline&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writerow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"registrar"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"expiryDate"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"daysToExpiry"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"sslDaysLeft"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"dmarc"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;ssl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ssl"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="n"&gt;dmarc&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dns"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"DMARC"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writerow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"registrar"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"expiryDate"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"daysToExpiry"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"daysLeft"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dmarc&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;alerts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"daysToExpiry"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="no"&gt;WARN_DAYS&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;alerts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{r['domain']}: domain expires in {r['daysToExpiry']} days"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;ssl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ssl"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"daysLeft"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"daysLeft"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="no"&gt;WARN_DAYS&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;alerts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{r['domain']}: SSL expires in {ssl['daysLeft']} days"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;alerts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="s2"&gt;"nothing expires in the next 30 days"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;alerts&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The non-zero exit code makes it easy to plug into cron, CI or any monitor that alerts on a failed job.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: run it daily and see only what changed
&lt;/h2&gt;

&lt;p&gt;Add &lt;code&gt;onlyNew&lt;/code&gt; and a &lt;code&gt;monitorId&lt;/code&gt; to the payload:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"onlyNew"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"monitorId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"my-domains"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first run reports every domain as &lt;code&gt;"changes": ["new"]&lt;/code&gt; (that is your baseline). After that, each run outputs only domains whose registrar, expiry, status, nameservers, DNSSEC, DNS records or certificate changed, and the &lt;code&gt;changes&lt;/code&gt; field says what. Unchanged domains are not saved and not charged.&lt;/p&gt;

&lt;p&gt;I ran it twice today, 20 seconds apart, on &lt;code&gt;stripe.com&lt;/code&gt;, &lt;code&gt;python.org&lt;/code&gt; and a made-up name (&lt;code&gt;radar-sample-unregistered-7731.com&lt;/code&gt;). The second run returned 0 rows: nothing changed, so nothing was saved or charged.&lt;/p&gt;

&lt;p&gt;An earlier version flagged &lt;code&gt;stripe.com&lt;/code&gt; as changed on every run because its load-balanced apex A records rotate between requests. Now A/AAAA records are ignored in only-new mode unless you set &lt;code&gt;"trackIpChanges": true&lt;/code&gt;, so the report stays quiet for normal load-balancer churn.&lt;/p&gt;

&lt;p&gt;To schedule it, either save the input as a task in Apify and add a schedule (&lt;code&gt;0 7 * * *&lt;/code&gt;) with an email/Slack/webhook integration, or keep the script above in your own cron:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;0 7 * * * &lt;span class="nb"&gt;cd&lt;/span&gt; /home/me/domains &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;xxx python3 check.py &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; check.log 2&lt;span class="o"&gt;&amp;gt;&amp;amp;&lt;/span&gt;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Not on Apify? Plain REST
&lt;/h2&gt;

&lt;p&gt;The same lookup is also listed as a REST API on RapidAPI and API.market (search for "Domain WHOIS Lookup API"), which is handy if you already have a key there. The Apify route above is the one I tested for this post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Not every TLD has RDAP.&lt;/strong&gt; A few ccTLD registries do not publish RDAP, and some rate-limit heavy use. Those domains are skipped, not charged, and listed in the &lt;code&gt;RUN_SUMMARY&lt;/code&gt; record with &lt;code&gt;lastError&lt;/code&gt;. Lower &lt;code&gt;maxConcurrency&lt;/code&gt; if you hit limits.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Unregistered domains come back as a row with &lt;code&gt;"available": true, "registered": false&lt;/code&gt;&lt;/strong&gt; (from an RDAP 404 plus DNS). The made-up name above was reported that way. It is a quick availability hint, not a registrar quote, so check price and premium status at your registrar.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;expiringWithinDays&lt;/code&gt;&lt;/strong&gt; keeps only domains whose registration or certificate ends within N days, if you want the alert list straight from the API.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use the registered domain&lt;/strong&gt; (&lt;code&gt;example.co.uk&lt;/code&gt;), not a subdomain. URLs are accepted; the host is used.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;DNS is live.&lt;/strong&gt; Values come from the resolver at run time. MX, TXT, NS and CNAME changes are reported in only-new mode; A/AAAA only with &lt;code&gt;trackIpChanges&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No personal data.&lt;/strong&gt; Registrant names, emails and addresses are never collected. If you need those, this is the wrong tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I have timed small lists, not huge ones.&lt;/strong&gt; The 4-domain run above took 0.7 s of lookups at concurrency 5. Big lists scale with registry rate limits, so start with a few hundred.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;Pay per result: $1.50 per 1,000 domains saved, no start fee, Apify compute included. The 4-domain run above cost $0.006, and in only-new mode you pay only for domains that changed. Apify's free plan includes a small monthly credit, enough to try it on your own list.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because I wanted one daily "what changed on my domains" report instead of four libraries and a calendar reminder. If a TLD does not work for you, or you need a field I do not return yet, leave a comment here or open an issue on the actor page and I will look at it.&lt;/p&gt;

&lt;p&gt;Next in this series: chaining it after a lead list (for example a Google Maps export) to add domain age, mail provider and SPF/DMARC to each company.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/domain-whois-dns-lookup?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/domain-whois-dns-lookup&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>devops</category>
      <category>security</category>
      <category>api</category>
    </item>
    <item>
      <title>8 free MCP tools for AI agents, no API key: web to Markdown, CVEs, domains, papers</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Fri, 06 Nov 2026 20:25:03 +0000</pubDate>
      <link>https://dev.to/martinpython/8-free-mcp-tools-for-ai-agents-no-api-key-web-to-markdown-cves-domains-papers-11i4</link>
      <guid>https://dev.to/martinpython/8-free-mcp-tools-for-ai-agents-no-api-key-web-to-markdown-cves-domains-papers-11i4</guid>
      <description>&lt;p&gt;Most MCP servers you find in directories want an API key, a local install, or both. I wanted the opposite for a few boring-but-useful lookups my agents do all day: read a web page as Markdown, check a domain, look up CVEs for a package version, search papers. So I put 8 of them behind one remote MCP URL that works &lt;strong&gt;without a key&lt;/strong&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;https://2-29-135-242.sslip.io/mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It is a streamable-HTTP MCP server, stateless JSON, read-only tools. Each IP gets 25 free calls per UTC day. Above that, every tool is also available as a pay-per-call x402 endpoint (USDC), charged only when it returns results.&lt;/p&gt;

&lt;h2&gt;
  
  
  The 8 tools
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;th&gt;Typical use&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;web_read&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;the page itself&lt;/td&gt;
&lt;td&gt;URL to clean Markdown for RAG or summaries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;domain_intel&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;RDAP + DNS + TLS&lt;/td&gt;
&lt;td&gt;registrar, expiry, nameservers, MX, SPF/DMARC, cert expiry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;email_auth&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;DNS&lt;/td&gt;
&lt;td&gt;SPF / DKIM / DMARC / MX check before cold email or vendor review&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;osv_vulns&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;OSV.dev + CISA KEV + EPSS&lt;/td&gt;
&lt;td&gt;known vulns for &lt;code&gt;package@version&lt;/code&gt;, fixed versions, exploit odds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;literature_search&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;OpenAlex + Crossref&lt;/td&gt;
&lt;td&gt;papers, DOIs, citations, open-access links&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;federal_register&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;federalregister.gov&lt;/td&gt;
&lt;td&gt;US final/proposed rules and notices&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;clinical_trials&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;ClinicalTrials.gov v2&lt;/td&gt;
&lt;td&gt;trials by condition, drug, sponsor, status&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;prediction_markets&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Polymarket + Kalshi&lt;/td&gt;
&lt;td&gt;live odds for an event in one call&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;All sources are official or open APIs. No logins and no scraping behind walls.&lt;/p&gt;

&lt;h2&gt;
  
  
  Connect it
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Claude Code&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;claude mcp add --transport http radar-tools https://2-29-135-242.sslip.io/mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Claude Desktop / Cursor / Windsurf&lt;/strong&gt; (&lt;code&gt;mcpServers&lt;/code&gt; JSON)&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"mcpServers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"radar-tools"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://2-29-135-242.sslip.io/mcp"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It is also in the official MCP Registry as &lt;code&gt;io.sslip.2-29-135-242/radar-agent-tools&lt;/code&gt;, so registry-aware clients can find it by name.&lt;/p&gt;

&lt;h2&gt;
  
  
  Raw call with curl
&lt;/h2&gt;

&lt;p&gt;MCP is just JSON-RPC over HTTP, so you can test it without any client:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s https://2-29-135-242.sslip.io/mcp \
  -H &lt;span class="s1"&gt;'content-type: application/json'&lt;/span&gt; \
  -H &lt;span class="s1"&gt;'accept: application/json, text/event-stream'&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"jsonrpc":"2.0","id":1,"method":"tools/call",
       "params":{"name":"osv_vulns",
                 "arguments":{"package":"requests","ecosystem":"PyPI","version":"2.25.0"}}}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Real output (trimmed), run on 2026-11-06:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"count"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"items"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"GHSA-j8r2-6x86-q33q"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"cves"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"CVE-2023-32681"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"summary"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Unintended leak of Proxy-Authorization header in requests"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"severity"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"MODERATE"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"fixedVersions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"2.31.0"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"knownExploited"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"epss"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.02974&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"upgradeToFixAll"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2.33.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"upgradeVerified"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="err"&gt;...&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The part I use most is &lt;code&gt;upgradeToFixAll&lt;/code&gt;: one version that fixes every listed advisory, re-checked on OSV so the suggested version itself has no known advisories. So an agent can open a single upgrade PR instead of reasoning about 8 advisories.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;domain_intel&lt;/code&gt; on &lt;code&gt;dev.to&lt;/code&gt; returns RDAP + DNS in one call (trimmed):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"dev.to"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
 &lt;span class="nl"&gt;"rdap"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"registrar"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Tonic"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"created"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2016-04-05T02:27:16.000Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="nl"&gt;"expires"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2027-04-05T02:27:16.000Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="nl"&gt;"nameservers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"jill.ns.cloudflare.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"josh.ns.cloudflare.com"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
 &lt;span class="nl"&gt;"dns"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"MX"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"1 smtp.google.com."&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
         &lt;span class="nl"&gt;"DMARC"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"v=DMARC1; p=reject; sp=reject; pct=100; ..."&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Prompts that work well
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;"Check &lt;code&gt;requests==2.25.0&lt;/code&gt; and &lt;code&gt;urllib3==1.26.4&lt;/code&gt; for known vulnerabilities and tell me the smallest safe upgrade."&lt;/li&gt;
&lt;li&gt;"Read &lt;a href="https://example.com/pricing" rel="noopener noreferrer"&gt;https://example.com/pricing&lt;/a&gt; and summarise the plans as a table."&lt;/li&gt;
&lt;li&gt;"When does the domain of this vendor expire, and is their email protected by DMARC?"&lt;/li&gt;
&lt;li&gt;"Find 5 recent papers on retrieval-augmented generation evaluation with DOIs."&lt;/li&gt;
&lt;li&gt;"What are the current odds on Polymarket and Kalshi for the next Fed rate decision?"&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits, honestly
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;25 free calls per IP per day, 1,500 free calls per day in total (it is one small server).&lt;/li&gt;
&lt;li&gt;Read-only. No browser rendering for heavy JavaScript apps in &lt;code&gt;web_read&lt;/code&gt;; it fetches the HTML and extracts the main content.&lt;/li&gt;
&lt;li&gt;Paid use is x402 (USDC on Base) at &lt;code&gt;https://2-29-135-242.sslip.io/x402/&amp;lt;tool&amp;gt;&lt;/code&gt;. If you would rather pay per result with a normal account, the bigger versions run as Apify Actors under &lt;code&gt;martin10291&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If a tool is missing a field you need, leave a comment with the call you made and what you expected. I read them.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>mcp</category>
      <category>ai</category>
      <category>api</category>
      <category>python</category>
    </item>
    <item>
      <title>Track new jobs at any Greenhouse, Lever or Ashby company daily with Python</title>
      <dc:creator>Martin Petrov</dc:creator>
      <pubDate>Thu, 05 Nov 2026 20:18:31 +0000</pubDate>
      <link>https://dev.to/martinpython/track-new-jobs-at-any-greenhouse-lever-or-ashby-company-daily-with-python-5bdf</link>
      <guid>https://dev.to/martinpython/track-new-jobs-at-any-greenhouse-lever-or-ashby-company-daily-with-python-5bdf</guid>
      <description>&lt;p&gt;You want to know the day a company you care about opens a new role, and checking ten careers pages by hand every morning is not a plan.&lt;/p&gt;

&lt;p&gt;Maybe you are a job seeker with a short list of dream companies. Maybe you sell to engineering teams and "they just opened 4 data engineer roles" is your best buying signal. Maybe you run a niche job board. In every case the data is public, but it is spread across different applicant tracking systems (ATS), each with its own page layout and its own JSON shape.&lt;/p&gt;

&lt;p&gt;I built a small tool for this, and in this post I show how to use it from Python: pull jobs from Greenhouse, Lever and Ashby (plus Workday and six other ATS) in one call, save them to CSV, and run it every day so you only see &lt;strong&gt;new&lt;/strong&gt; postings.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the usual approach hurts
&lt;/h2&gt;

&lt;p&gt;The usual path looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Scrape the careers page HTML. It breaks on the next redesign, and big career sites are rendered with JavaScript.&lt;/li&gt;
&lt;li&gt;Find the ATS behind it. Greenhouse has &lt;code&gt;boards-api.greenhouse.io&lt;/code&gt;, Lever has &lt;code&gt;api.lever.co&lt;/code&gt;, Ashby has its own posting API, Workday has its own paginated endpoint per tenant. Each one returns different field names for the same thing.&lt;/li&gt;
&lt;li&gt;Write a normaliser per ATS, a dedupe step, and some state so tomorrow's run knows what you already saw.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Steps 2 and 3 are where the time goes. None of it is hard, but it is a weekend of glue code, and then you have to maintain it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I built
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://apify.com/martin10291/ats-jobs-aggregator" rel="noopener noreferrer"&gt;ATS Jobs Scraper&lt;/a&gt; is an Apify actor that reads the public job-board endpoints of 10 ATS (Workday, Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor, Breezy HR) and returns every job in &lt;strong&gt;one schema&lt;/strong&gt;: title, department, location, remote flag, salary range when the company publishes one, posting date and apply link. No login, no browser, no CAPTCHA, because it only uses the endpoints the careers pages themselves call.&lt;/p&gt;

&lt;p&gt;You give it a list like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;greenhouse:stripe
lever:palantir
ashby:ramp
https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A plain careers URL works too (&lt;code&gt;https://boards.greenhouse.io/stripe&lt;/code&gt;, &lt;code&gt;https://jobs.lever.co/palantir&lt;/code&gt;), and for most ATS a bare company slug is auto-detected.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: get an Apify token
&lt;/h2&gt;

&lt;p&gt;Create a free Apify account and copy your API token from &lt;strong&gt;Settings -&amp;gt; API &amp;amp; Integrations&lt;/strong&gt;. Put it in an environment variable so it never ends up in your code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"your-token-here"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: one call with curl
&lt;/h2&gt;

&lt;p&gt;The &lt;code&gt;run-sync-get-dataset-items&lt;/code&gt; endpoint starts the actor, waits for it to finish and returns the results in the same HTTP response:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s -X POST \
  &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator/run-sync-get-dataset-items?token=$APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{
    "companies": ["greenhouse:stripe", "lever:palantir", "ashby:ramp"],
    "titleKeywords": ["engineer"],
    "maxJobsPerCompany": 2,
    "maxItems": 6,
    "includeDescription": false
  }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I ran exactly this input today (2026-11-05). The run took 5.5 seconds. These are the 6 rows it returned, trimmed to the main fields:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;company&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;th&gt;location&lt;/th&gt;
&lt;th&gt;postedAt&lt;/th&gt;
&lt;th&gt;salary&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Stripe&lt;/td&gt;
&lt;td&gt;AI Engineer&lt;/td&gt;
&lt;td&gt;Chicago&lt;/td&gt;
&lt;td&gt;2026-10-07&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Stripe&lt;/td&gt;
&lt;td&gt;Android BSP Engineer&lt;/td&gt;
&lt;td&gt;Taipei, Taiwan&lt;/td&gt;
&lt;td&gt;2026-09-29&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Palantir&lt;/td&gt;
&lt;td&gt;Backend Software Engineer - Application Development&lt;/td&gt;
&lt;td&gt;London, United Kingdom&lt;/td&gt;
&lt;td&gt;2024-03-11&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Palantir&lt;/td&gt;
&lt;td&gt;Backend Software Engineer - Application Development&lt;/td&gt;
&lt;td&gt;New York, NY&lt;/td&gt;
&lt;td&gt;2024-03-11&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ramp&lt;/td&gt;
&lt;td&gt;Security Engineer, Cloud&lt;/td&gt;
&lt;td&gt;New York, NY (HQ)&lt;/td&gt;
&lt;td&gt;2026-04-07&lt;/td&gt;
&lt;td&gt;211,400-290,600&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ramp&lt;/td&gt;
&lt;td&gt;Mobile Engineer, Android&lt;/td&gt;
&lt;td&gt;New York, NY (HQ)&lt;/td&gt;
&lt;td&gt;2025-07-31&lt;/td&gt;
&lt;td&gt;189,000-330,000&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;And one full row, exactly as it came back:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"greenhouse:stripe:8144196"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"ats"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"greenhouse"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"companySlug"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"stripe"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"company"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Stripe"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"title"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"AI Engineer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"department"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"1155 SA Transformation"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Chicago"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"locations"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"US"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"isRemote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"salaryMin"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"salaryMax"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"salaryCurrency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"postedAt"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-10-07T15:39:36Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://stripe.com/jobs/search?gh_jid=8144196"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"applyUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://stripe.com/jobs/search?gh_jid=8144196"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"scrapedAt"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-05T20:16:52Z"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The run also writes a &lt;code&gt;RUN_SUMMARY&lt;/code&gt; record to its key-value store. Today it said Stripe had 763 open jobs, Palantir 308 and Ramp 170, and that 2 were saved from each because of my limits. That summary is handy when you want to chart how many roles a company has open over time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: the same thing in Python, saved to CSV
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator"&lt;/span&gt;
       &lt;span class="s2"&gt;"/run-sync-get-dataset-items"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"companies"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="s2"&gt;"greenhouse:stripe"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"lever:palantir"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"ashby:ramp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"titleKeywords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"engineer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"data"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"excludeKeywords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"intern"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"locations"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"London"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Berlin"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Remote"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"includeRemoteWithLocations"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"postedWithinDays"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"includeDescription"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"maxItems"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;TOKEN&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;jobs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{len(jobs)} jobs"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;fields&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"company"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"title"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"isRemote"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"salaryMin"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="s2"&gt;"salaryMax"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"salaryCurrency"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"postedAt"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"applyUrl"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"jobs.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;newline&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"utf-8"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;DictWriter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fieldnames&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;fields&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;extrasaction&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"ignore"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writeheader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;writerows&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The filters run inside the actor, so you only pay for the rows you keep. &lt;code&gt;postedWithinDays&lt;/code&gt; and &lt;code&gt;minSalary&lt;/code&gt; are useful here; note that &lt;code&gt;minSalary&lt;/code&gt; drops jobs that publish no salary at all.&lt;/p&gt;

&lt;p&gt;If you want the rows in Google Sheets instead of a CSV, &lt;code&gt;gspread&lt;/code&gt; takes the same list:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;gspread&lt;/span&gt;

&lt;span class="n"&gt;gc&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;gspread&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;service_account&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;filename&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"service_account.json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;ws&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;gc&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Job feed"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sheet1&lt;/span&gt;
&lt;span class="n"&gt;ws&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append_rows&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;fields&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 4: run it daily and get only new jobs
&lt;/h2&gt;

&lt;p&gt;This is the part that saves the most glue code. Add one flag:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"onlyNewJobs"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;
&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"postedWithinDays"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;  &lt;span class="c1"&gt;# let the state decide what is new&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With &lt;code&gt;onlyNewJobs&lt;/code&gt; the actor stores the IDs it has already returned for that exact company list in a named key-value store in &lt;strong&gt;your&lt;/strong&gt; Apify account. The next run with the same list returns only jobs that were not there before. The first run returns everything (that is your baseline); after that you get the daily delta.&lt;/p&gt;

&lt;p&gt;Two ways to schedule it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apify Schedules&lt;/strong&gt;: open the actor, save your input as a task, then add a schedule (for example &lt;code&gt;0 7 * * *&lt;/code&gt;). Connect the task to a webhook, Slack, email or Zapier/Make so new jobs land where you work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your own cron&lt;/strong&gt;: keep the Python script above and run it from cron or GitHub Actions:
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;0 7 * * * &lt;span class="nb"&gt;cd&lt;/span&gt; /home/me/jobfeed &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;APIFY_TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;xxx python3 feed.py &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; feed.log 2&lt;span class="o"&gt;&amp;gt;&amp;amp;&lt;/span&gt;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you change the company list, the state key changes too, so a new list starts a fresh baseline.&lt;/p&gt;

&lt;h2&gt;
  
  
  Not on Apify? Same data as a plain REST API
&lt;/h2&gt;

&lt;p&gt;I also list the same scraper as a regular REST API on RapidAPI and API.market (search for "Jobs API - Workday, Greenhouse, Lever, Ashby"). That route makes sense if you already have a RapidAPI key and want one HTTP call without an Apify account. The Apify route above is the most direct one and the one I tested for this post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It only sees companies you name.&lt;/strong&gt; It is not a search engine over all jobs on the internet. You bring the list; the tool reads those boards.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Salary is only as good as the source.&lt;/strong&gt; Ashby, Lever and Recruitee often return structured ranges (like the Ramp rows above). For other ATS the actor tries to detect a range in the text, and many postings simply have none, like the Stripe rows above.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fields mean what the ATS says.&lt;/strong&gt; Palantir's backend roles show &lt;code&gt;postedAt&lt;/code&gt; 2024-03-11 because those are evergreen postings that stay open for years. Stripe's &lt;code&gt;location&lt;/code&gt; and &lt;code&gt;locations&lt;/code&gt; can differ because Greenhouse stores them separately. I keep the original values instead of guessing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Workday tenants are slower.&lt;/strong&gt; Big Workday sites have thousands of roles and paginate, so set &lt;code&gt;maxJobsPerCompany&lt;/code&gt; or keyword filters if you only need a slice.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ATS changes happen.&lt;/strong&gt; If a company moves from Lever to Ashby, its old slug returns 0 jobs. The &lt;code&gt;RUN_SUMMARY&lt;/code&gt; shows jobs per company, so a sudden 0 is easy to spot.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Price
&lt;/h2&gt;

&lt;p&gt;Pay per result: you pay only for jobs saved after your filters, with no start fee. It is $1 per 1,000 jobs until 13 November 2026 and $2 per 1,000 jobs after that. With pay-per-result pricing the Apify compute is included, so the 6-row run above cost $0.006 at today's price. Apify's free plan includes a small monthly credit, enough to try it on your own company list.&lt;/p&gt;

&lt;h2&gt;
  
  
  Wrap-up
&lt;/h2&gt;

&lt;p&gt;I built this because I wanted one feed of "new roles at these companies" without maintaining ten scrapers. If you try it and a company's board does not parse, or you need an ATS I do not cover yet, leave a comment or open an issue on the actor page and I will look at it.&lt;/p&gt;

&lt;p&gt;Next in this series: turning the daily new-jobs feed into a hiring-signal list for sales (which companies just opened roles in a given department).&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Try it in your browser first, no code:&lt;/strong&gt; &lt;a href="https://radarbg.com/apis/ats-jobs-aggregator?ref=devto" rel="noopener noreferrer"&gt;radarbg.com/apis/ats-jobs-aggregator&lt;/a&gt; - free live tries, plus free online tools at &lt;a href="https://radarbg.com/?ref=devto" rel="noopener noreferrer"&gt;radarbg.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build the tool used in this post, and the article was written with AI assistance. If a code sample does not work for you, tell me in the comments and I will fix it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>jobs</category>
      <category>automation</category>
    </item>
  </channel>
</rss>
