<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en_GB"><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="http://blog.gnclmorais.com/feed.xml" rel="self" type="application/atom+xml" /><link href="http://blog.gnclmorais.com/" rel="alternate" type="text/html" hreflang="en_GB" /><updated>2026-02-18T17:14:52+00:00</updated><id>http://blog.gnclmorais.com/feed.xml</id><title type="html">Crushed by Code</title><subtitle>A (rather technical) blog about (mostly) web development.</subtitle><entry><title type="html">Simple window management for interviews</title><link href="http://blog.gnclmorais.com/simple-window-management-for-interviews" rel="alternate" type="text/html" title="Simple window management for interviews" /><published>2022-10-16T00:00:00+00:00</published><updated>2022-10-16T00:00:00+00:00</updated><id>http://blog.gnclmorais.com/simple-window-management-for-interviews</id><content type="html" xml:base="http://blog.gnclmorais.com/simple-window-management-for-interviews"><![CDATA[<p>In the two years, I’ve spent a lot of time performing interviews at my current job — not just for my product group, but for other UI teams as well. Along the way, there are some things I’ve picked that make the whole process easier, things that you wouldn’t necessarily read about.</p>

<p>One of these things is window management. Working at a fully remote company means you might spend countless hours on video calls for meetings, interviews, or even social events. Sometimes you’ll have a Zoom/Skype/Slack/whatever window open and that is great. Other times you’ll have to juggle video, notes, documentation, etc.</p>

<p>Regarding remote interviews, I think I hit my sweet spot:</p>

<p><img src="/images/posts/2022-07-15--interview-window-setup.png" alt="Monitor showing top half with video call window (with three people, the middle one being the interviewee), bottom left quarter with interview notes, and bottom right corner with interview script." /></p>

<h2 id="video-at-the-top">Video at the top</h2>

<p>Throughout this Covid-induced period of constant video calls, I’ve found the trick of aligning the video thumbnails of people on the call with the camera, so it looks like you are looking at them when you look at their video. This shows more presence on your part, you don’t look constantly distracted (like people with two monitors where the camera is not on the main monitor they look at).</p>

<h2 id="notes-script-and-comms">Notes, script and comms</h2>

<p>The bottom half of the screen can be split as you want. In my case, we have an interview script that guides us through the sections we have to evaluate and allows us to provide less biased and more standard interviews (i.e. everyone gets pretty much the same questions). I also take notes about the interview as it goes along, ideally without looking at the keyboard (to keep a stronger feeling of engagement). We usually have two interviewers per interview, so I also find it useful to have Slack open so the other interviewer and I can quickly sync about the interview pace or last minute change of plans.</p>

<hr />

<h2 id="credits">Credits</h2>

<p>Thank you to <a href="https://www.behance.net/shillin">Dmytro Shylin</a> for the <a href="https://www.figma.com/community/file/897520900352352669">Figma monitor mockup</a>.</p>]]></content><author><name></name></author><category term="interviewing" /><category term="remote" /><summary type="html"><![CDATA[In the two years, I’ve spent a lot of time performing interviews at my current job — not just for my product group, but for other UI teams as well. Along the way, there are some things I’ve picked that make the whole process easier, things that you wouldn’t necessarily read about.]]></summary></entry><entry><title type="html">Run copied CLI commands with a leading “$” sign safer</title><link href="http://blog.gnclmorais.com/run-copied-cli-commands-with-a-leading-dollar-sign-safer" rel="alternate" type="text/html" title="Run copied CLI commands with a leading “$” sign safer" /><published>2021-07-20T10:00:00+00:00</published><updated>2021-07-20T10:00:00+00:00</updated><id>http://blog.gnclmorais.com/run-copied-cli-commands-with-a-leading-dollar-sign-safer</id><content type="html" xml:base="http://blog.gnclmorais.com/run-copied-cli-commands-with-a-leading-dollar-sign-safer"><![CDATA[<p>I recently came across <a href="https://www.stefanjudis.com/blog/how-to-run-commands-with-a-leading-usd-sign">Stefan’s tip</a> on how to run copied commands with a leading dollar sign and I thought it was a great idea! However, this heads-up caught my eye:</p>

<blockquote>
  <p>Use at your own risk. 🤓 Before implementing any functionality that makes running copied code easier, be aware that the internet’s a bad place. There’s always a chance that a command has malicious intent or <a href="http://thejh.net/misc/website-terminal-copy-paste">even includes hidden commands</a>.</p>
</blockquote>

<p>This made me think: maybe I can build upon his example and add a small layer of protection, to prevent accidental errors when working a bit absentmindedly — we all do that from time to time.</p>

<h2 id="ask-for-confirmation-before">Ask for confirmation before</h2>

<p>Follow his tutorial and, when you get to the part when you’re writing the script, instead of simply executing the commands it gets, ask the user for <em>explicit</em> confirmation before proceeding. Use this as the content of your <code class="language-plaintext highlighter-rouge">$</code> executable:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/zsh</span>
<span class="nb">read</span> <span class="nt">-p</span> <span class="s2">"Are you sure you? [y/N]: "</span> <span class="nt">-n</span> 1 <span class="nt">-r</span>
<span class="nb">echo</span> <span class="c"># move to a new line</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$REPLY</span> <span class="o">=</span>~ ^[Yy]<span class="nv">$ </span><span class="o">]]</span>
<span class="k">then
    </span><span class="nb">exec</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span>
<span class="k">fi</span>
</code></pre></div></div>

<p>I <a href="https://stackoverflow.com/a/1885534/590525">used someone’s else suggestion</a> in order to ask for confirmation, but this is a simple way of adding a small confirmation step. Anything that is not <code class="language-plaintext highlighter-rouge">y</code> or <code class="language-plaintext highlighter-rouge">Y</code> will skip the execution. This will hopefully help with the <a href="http://thejh.net/misc/website-terminal-copy-paste">hidden commands issue</a> Stefan made me aware of.</p>]]></content><author><name></name></author><category term="bash" /><summary type="html"><![CDATA[I recently came across Stefan’s tip on how to run copied commands with a leading dollar sign and I thought it was a great idea! However, this heads-up caught my eye:]]></summary></entry><entry><title type="html">Build and deploy a web scraper in JavaScript with Puppeteer, part 3</title><link href="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-iii" rel="alternate" type="text/html" title="Build and deploy a web scraper in JavaScript with Puppeteer, part 3" /><published>2021-07-18T09:00:00+00:00</published><updated>2021-07-18T09:00:00+00:00</updated><id>http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-iii</id><content type="html" xml:base="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-iii"><![CDATA[<p><em>This is the third post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series:</em>
`</p>
<ol>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper">Write the scraping script</a></li>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-ii">Add SMS notifications</a></li>
  <li>Deploy and run it automatically <em>(this post)</em></li>
</ol>

<hr />

<p>Welcome back! Now that we have a script that checks a page and we added SMS notifications, let’s make sure we get this up and running — executing our script regularly.</p>

<h2 id="create-a-heroku-account">Create a Heroku account</h2>

<p>This tutorial relies on Heroku for the simple fact that it allows us to be abstracted from most server-side hassle and we can focus on the project at hands. If you have the project we’ve been building tracked with Git, this will be smooth.</p>

<p>Make sure you have a Heroku account and install their <a href="https://devcenter.heroku.com/articles/heroku-cli#download-and-install">CLI</a>. Then <a href="https://devcenter.heroku.com/articles/heroku-cli#getting-started">log in</a> through it so that you have access to your Heroku account using terminal commands.</p>

<h2 id="deploy-your-app">Deploy your app</h2>

<p>After making sure you’re on the project folder, run <code class="language-plaintext highlighter-rouge">heroku create</code> to make a new empty application on Heroku. This doesn’t deploy your code yet.</p>

<p>Before we send our code to Heroku, make sure you have the following lines on your <code class="language-plaintext highlighter-rouge">package.json</code> — it will help us to run our script with the last amount of work:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"start"</span><span class="p">:</span><span class="w"> </span><span class="s2">"node index.js"</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>To send your code to be run by Heroku, execute the following: <code class="language-plaintext highlighter-rouge">git push heroku main</code>. This will take a while and will output a lot of text, keeping you up to date with the remote state of the dyno Heroku is setting up for you. At the end, you’re greeted with a message with a link, something like <code class="language-plaintext highlighter-rouge">https://hidden-socks-12321.herokuapp.com/ deployed to Heroku</code>. In our case, this is not important since we don’t have a page to look at — we’ll get an SMS notifications whenever we find what we want.</p>

<h2 id="schedule-regular-jobs">Schedule regular jobs</h2>

<p>Our script is on Heroku, so we’re just missing one last step: making sure it runs regularly. Head out to <a href="https://dashboard.heroku.com/apps">your dashboard</a> where you’ll see all your current applications. Find the one we just created and select it.</p>

<p>On you app’s page, go to “Resources” then click “Find more add-ons”. You’ll be greeted with a long page full of possible addons, but we’re looking for one on the <a href="https://elements.heroku.com/addons#dynos">Dynos</a> section. Find <a href="https://elements.heroku.com/addons/scheduler">Heroku Scheduler</a> and click on it. You’ll see a button near the top right corner of your screen that says “Install Heroku Scheduler”, press it and, on the next screen, type the name of your app and click “Submit Order Form” — don’t worry, this addon is free.</p>

<p>The addon is now enabled and we can see the options we have available. You should be back on your app’s page, on the Resources tab, where you should see your newly installed addons. Click on it to access its configuration page.</p>

<p>On this new page, click “Create job” to show a right-side panel where you can pick if you want to run your script every:</p>
<ul>
  <li>10 minutes</li>
  <li>Every hour at 00, 10, 20, 30, 40 or 50 minutes in the hour</li>
  <li>Every day at a certain time (in 30 minutes increments)</li>
</ul>

<p>Pick the frequence you want, pass the command we have to run the script (<code class="language-plaintext highlighter-rouge">npm start</code>) and click “Save Job” at the bottom.</p>

<h2 id="one-last-step-a-missing-buildpack">One last step, a missing buildpack</h2>

<p>Heroku has this concept of <em>buildpacks</em> that they define as “(…) scripts that are run when your app is deployed. They are used to install dependencies for your app and configure your environment.”</p>

<p>For this particular project, there is one we must add to have access to Puppeteer, created by <a href="https://github.com/jontewks">Jon Tewksbury</a>. It will help Heroku installing all the necessary dependencies (like Chrome) so that our script runs without problems.</p>

<p>On your app’s dashboard page, go to the Settings tab and scroll down to the section Buildpacks. Click “Add buildpack” and paste <code class="language-plaintext highlighter-rouge">https://github.com/jontewks/puppeteer-heroku-buildpack</code> in the text input of the modal you get. Click “Save changes” and you’re ready to go.</p>

<p>Buildpacks are used the next time our apps are deployed, so just to be sure everything is in place, let’s trigger a redeploy of our app by pushing an empty commit:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git commit <span class="nt">--allow-empty</span> <span class="nt">-m</span> <span class="s2">"Trigger deploy after buildpack"</span> <span class="o">&amp;&amp;</span> git push heroku main
</code></pre></div></div>

<h2 id="conclusion">Conclusion</h2>

<p>That’s it! You have now created, deployed and scheduled a web scraper that notifies you with an SMS whenever it finds what it is looking for! There are numerous other ways to do what we’ve accomplished, however, I’ve found this the most <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto</a>-like way of spinning something up whenever I have the need to automate checking up on a website. I’ve optimised for simplicity and speed of MVP on these blogs, not for flexibility or power.</p>

<p>As a last tip, if you are looking for logs to see what is happening with your app from time to time or you’re trying to debug something with it, on your app’s dashboard you should see a “More” button on the top right corner. Click on it to expand the options available and you’ll see a “View logs” item. That’s where you can take a closer look at the logs Heroku collects from your running applications.</p>]]></content><author><name></name></author><category term="javascript" /><category term="puppeteer" /><summary type="html"><![CDATA[This is the third post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series: ` Write the scraping script Add SMS notifications Deploy and run it automatically (this post)]]></summary></entry><entry><title type="html">Build and deploy a web scraper in JavaScript with Puppeteer, part 2</title><link href="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-ii" rel="alternate" type="text/html" title="Build and deploy a web scraper in JavaScript with Puppeteer, part 2" /><published>2021-06-18T09:00:00+00:00</published><updated>2021-06-18T09:00:00+00:00</updated><id>http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-ii</id><content type="html" xml:base="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-ii"><![CDATA[<p><em>This is the second post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series:</em></p>

<ol>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper">Write the scraping script</a></li>
  <li>Add SMS notifications <em>(this post)</em></li>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-iii">Deploy and run it automatically</a></li>
</ol>

<hr />

<p>Welcome back! Now that we have a basic script that checks for the data on a page, let’s add an SMS notification that will inform us of the current headline we find.</p>

<h2 id="create-a-twilio-account">Create a Twilio account</h2>

<p>We’re going to use Twilio for this, they have a free API that we can quickly reach out to and send a few SMS through it. Head out to https://twilio.com/try-twilio and create your account (if you already have one, read along).</p>

<p>After creating your account and logging in, we want to create a new project (sometimes also called <em>account</em> on their platform). These are containers for your applications (nothing to do with Docker, though).</p>

<p>Look for the <em>Create New Account</em> button or go straight to <a href="https://www.twilio.com/console/projects/create">twilio.com/console/projects/create</a>. Pick a name for your account and click <em>Verify</em>:</p>

<p><img src="/images/posts/2021-06-18--01--new-account.png" alt="New account/project on Twilio" /></p>

<p>The next screen asks you to verify yourself. Yeah, it’s a bit annoying, but this also helps them to prevent some less benevolent agents from using their platform, so it’s cool.</p>

<p><img src="/images/posts/2021-06-18--02--verify-phone-number.png" alt="New account/project on Twilio" /></p>

<p>Write your phone number down, go to the next page, type the verification code you got on your phone, submit and you’re almost there! The next screen asks us for some goals, in order to send us to the right place and the right features we intend to use. I’ve picked the following for this example:</p>

<p><img src="/images/posts/2021-06-18--03--answers-to-twilio.png" alt="Answers to Twilio welcome screen" /></p>

<p>After this, we are finally greeted with our account dashboard and… there is a lot of stuff. 😅 Luckily, we’re only here for one thing: a phone number to send SMS from. If you scroll down a bit, you should see a <em>Get a Twilio trial phone number</em> header. Click the button below it, <em>Get a trial phone number</em>. You should get a modal with a suggested phone number.</p>

<p><img src="/images/posts/2021-06-18--04--suggested-phone-number.png" alt="Modal with a suggested phone number" /></p>

<p>There is nothing special with the phone number we’re looking for, so grab the first one you get by clicking on <em>Choose this Number</em>. Press <em>Done</em> on the next modal and we have now a number! We’re very close to being able to send messages…</p>

<h3 id="small-gotcha">Small gotcha</h3>

<p>In order to prevent free accounts from being used to spam people, you’ll only be able tos end SMS to verified numbers. You can check the only one you got so far (the one you used to sign up) at <a href="https://www.twilio.com/console/phone-numbers/verified">twilio.com/console/phone-numbers/verified</a>. If there are other numbers you want to message through your script, you should add them here now.</p>

<h2 id="integrate-twilio-in-our-script">Integrate Twilio in our script</h2>

<p>We’re ready to use Twilio and send some SMS. To do that, we’ll need their npm package, so install it by running <code class="language-plaintext highlighter-rouge">npm install twilio</code> (if you have npm 5 of above, this should save the package on your <code class="language-plaintext highlighter-rouge">package.json</code>).</p>

<p>Start now by creating a separate file where we’ll put out notification code, <code class="language-plaintext highlighter-rouge">notify.js</code>. This will be our base code to send notifications:</p>

<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">twilio</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">twilio</span><span class="dl">'</span><span class="p">);</span>

<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
  <span class="nx">sendSMS</span><span class="p">(</span><span class="nx">msg</span><span class="p">,</span> <span class="nx">toNumber</span><span class="p">)</span> <span class="p">{</span>
    <span class="kd">const</span> <span class="nx">fromNumber</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">TWILLIO_PHONE_NUMBER</span><span class="p">;</span>
    <span class="kd">const</span> <span class="nx">accountSid</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">TWILLIO_ACCOUNT_SID</span><span class="p">;</span>
    <span class="kd">const</span> <span class="nx">authToken</span>  <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">TWILLIO_AUTH_TOKEN</span><span class="p">;</span>

    <span class="kd">const</span> <span class="nx">client</span> <span class="o">=</span> <span class="nx">twilio</span><span class="p">(</span><span class="nx">accountSid</span><span class="p">,</span> <span class="nx">authToken</span><span class="p">);</span>

    <span class="k">return</span> <span class="nx">client</span><span class="p">.</span><span class="nx">messages</span>
      <span class="p">.</span><span class="nx">create</span><span class="p">({</span>
         <span class="na">body</span><span class="p">:</span> <span class="nx">msg</span><span class="p">,</span>
         <span class="na">from</span><span class="p">:</span> <span class="nx">fromNumber</span><span class="p">,</span>
         <span class="na">to</span><span class="p">:</span> <span class="nx">toNumber</span><span class="p">,</span>
       <span class="p">})</span>
      <span class="p">.</span><span class="nx">then</span><span class="p">(</span><span class="nx">message</span> <span class="o">=&gt;</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">message</span><span class="p">.</span><span class="nx">sid</span><span class="p">))</span>
      <span class="p">.</span><span class="nx">done</span><span class="p">();</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>The code above is almost straight from <a href="https://www.twilio.com/docs/sms/quickstart/node">their documentation</a> (which is excellent, by the way!) and I’m always amazed about how little code you need to send SMS!</p>

<p>You can also notice how we have three lines accessing <code class="language-plaintext highlighter-rouge">process.env.*</code>. You do this in Node to access environment variables, i.e., values that you can set on the fly when you run the command. Think of it as function arguments, but for Node scripts.</p>

<p>With our notification module ready, go back to your <code class="language-plaintext highlighter-rouge">index.js</code> and we’ll import it to give it a spin:</p>

<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code> const puppeteer = require('puppeteer');
<span class="gi">+const { sendSMS } = require('./notify');
+
+const toNumber = process.env.MY_PHONE_NUMBER;
</span>
 (async () =&gt; {
   const browser = await puppeteer.launch({
<span class="p">@@ -19,7 +22,7 @@</span> const puppeteer = require('puppeteer');
   const header = await getText(firstArticle, 'h2');
   const summary = await getText(firstArticle, 'p:first-of-type');

-  console.log(`${header}\n${summary}`);
<span class="gi">+  sendSMS(`${header}\n${summary}`, toNumber);
</span>
   await browser.close();
 })();
</code></pre></div></div>

<p>The text we were passing to <code class="language-plaintext highlighter-rouge">console.log</code> we’ll not send to our phones with the new <code class="language-plaintext highlighter-rouge">sendSMS</code> method we created. Don’t forget to also get the number you are sending this message to (should be the same you used to register to Twilio) through an environment variable as well.</p>

<h2 id="run-the-code">Run the code</h2>

<p>We have everything in place now, there are just a few consideration to have before heading to a terminal and running all of this.</p>

<p>The <code class="language-plaintext highlighter-rouge">process.env.*</code> variables we set in our code must be provided by us in some way — I say this becuase we can do it in several ways, but we’ll follow the simplest. When running our <code class="language-plaintext highlighter-rouge">index.js</code> script, we’ll pass these environment variables inline. Here’s an example (make sure you use your own credentials that you get from <a href="https://www.twilio.com/console">the console page</a>):</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">TWILLIO_PHONE_NUMBER</span><span class="o">=</span><span class="s2">"+19293949596"</span> <span class="se">\</span>
<span class="nv">TWILLIO_ACCOUNT_SID</span><span class="o">=</span>ACDCadbfe2ce33c691a6dcfdce6e3617bb <span class="se">\</span>
<span class="nv">TWILLIO_AUTH_TOKEN</span><span class="o">=</span>face0a31ee17c4a2c9b3c0vfefeffa1f <span class="se">\</span>
<span class="nv">MY_PHONE_NUMBER</span><span class="o">=</span><span class="s2">"+447663342007"</span> <span class="se">\</span>
node index.js
</code></pre></div></div>

<p>The backslashes allow us to split a long command into multiple lines, for readability’s sake. Instead of harcoding this sensitive data into our code, we extracted it to configurable variables. This will permit us to easily set up an integration pipeline in the future that will run this script automatically, and also to make this project reusable by other people with their own Twilio credentials.</p>

<p>And that’s it for the time being! You now have a script that send you an SMS with dynamically fetched data.</p>

<p>See you around soon for the third part of this article…</p>]]></content><author><name></name></author><category term="javascript" /><category term="puppeteer" /><summary type="html"><![CDATA[This is the second post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series:]]></summary></entry><entry><title type="html">Build and deploy a web scraper in JavaScript with Puppeteer, part 1</title><link href="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper" rel="alternate" type="text/html" title="Build and deploy a web scraper in JavaScript with Puppeteer, part 1" /><published>2021-06-15T09:00:00+00:00</published><updated>2021-06-15T09:00:00+00:00</updated><id>http://blog.gnclmorais.com/build-and-deploy-a-web-scraper</id><content type="html" xml:base="http://blog.gnclmorais.com/build-and-deploy-a-web-scraper"><![CDATA[<p><em>This is the first post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series:</em></p>

<ol>
  <li>Write the scraping script <em>(this post)</em></li>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-ii">Add SMS notifications</a></li>
  <li><a href="https://blog.gnclmorais.com/build-and-deploy-a-web-scraper-part-iii">Deploy and run it automatically</a></li>
</ol>

<hr />

<p>I’ve had a few situations in the past where I was waiting for something to get updated on a website and just kept refreshing the page every so often… But when you don’t know when that update is going to happen, this can get tedious and hey, we’re programmers, we can build something to do this for us. 😃</p>

<p><em>“<a href="https://www.npmjs.com/package/puppeteer">Puppeteer</a> is a Node library which provides a high-level API to control Chrome”</em> and it’s the one I usually use just because it makes building a simple web scraper super simple. Let’s dig in and build a Minimum Viable Product that, for the sake of this example, grabs the top news from The New York Times’ <a href="https://www.nytimes.com/section/todayspaper"><em>Today’s Paper</em></a>.</p>

<h2 id="project-start">Project start</h2>

<p>Begin by creating a <code class="language-plaintext highlighter-rouge">package.json</code> that will hold the project’s dependencies. You can use <code class="language-plaintext highlighter-rouge">npm init</code> for this, but for simplicity’s sake, I’ll create a stripped-down version:</p>

<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// package.json</span>
<span class="p">{</span>
  <span class="dl">"</span><span class="s2">name</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">web-scraper-with-puppeteer</span><span class="dl">"</span><span class="p">,</span>
  <span class="dl">"</span><span class="s2">version</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">1.0.0</span><span class="dl">"</span><span class="p">,</span>
  <span class="dl">"</span><span class="s2">private</span><span class="dl">"</span><span class="p">:</span> <span class="kc">true</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Now we add our only dependency, Puppeteer. Run this on the terminal:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm <span class="nb">install </span>puppeteer
</code></pre></div></div>

<p>Your <code class="language-plaintext highlighter-rouge">package.json</code> has changed a bit now, here’s the difference:</p>

<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code> {
   "name": "web-scraper-with-puppeteer",
   "version": "1.0.0",
<span class="gd">-  "private": true
</span><span class="gi">+  "private": true,
+  "dependencies": {
+    "puppeteer": "^9.1.1"
+  }
</span> }
</code></pre></div></div>

<p>Let’s start with our main script now. Open up a brand new <code class="language-plaintext highlighter-rouge">index.js</code> and write the following:</p>

<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// index.js</span>
<span class="kd">const</span> <span class="nx">puppeteer</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">puppeteer</span><span class="dl">'</span><span class="p">);</span>

<span class="p">(</span><span class="k">async</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">browser</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">puppeteer</span><span class="p">.</span><span class="nx">launch</span><span class="p">({</span>
    <span class="na">headless</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span>
  <span class="p">});</span>
  <span class="kd">const</span> <span class="nx">page</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">browser</span><span class="p">.</span><span class="nx">newPage</span><span class="p">();</span>

  <span class="k">await</span> <span class="nx">page</span><span class="p">.</span><span class="nx">goto</span><span class="p">(</span>
    <span class="dl">'</span><span class="s1">https://nytimes.com/section/todayspaper</span><span class="dl">'</span>
  <span class="p">);</span>
  <span class="k">await</span> <span class="nx">browser</span><span class="p">.</span><span class="nx">close</span><span class="p">();</span>
<span class="p">})();</span>
</code></pre></div></div>

<p>For now, this is a simple script that you can run right now with <code class="language-plaintext highlighter-rouge">node index.js</code> in order to see if everything is going well so far. You should see a Chrome window opening up (because we specified <code class="language-plaintext highlighter-rouge">headless: false</code>) and closing as soon as the page stops loading. So far so good! Let’s now grab from the DOM the first article on the page.</p>

<p>Add the next lines to your script to grab the first article and just output its HTML, so we can see if we’re retrieving the right thing:</p>

<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   await page.goto(
     'https://nytimes.com/section/todayspaper'
   );
<span class="gi">+
+  const firstArticle = await page.$eval(
+    'article:first-of-type',
+    e =&gt; e.outerHTML
+  );
+
+  console.log(firstArticle);
+
</span>   await browser.close();
 })();
</code></pre></div></div>

<p>Run your script with <code class="language-plaintext highlighter-rouge">node index.js</code> and you should see a lot of HTML inside an <code class="language-plaintext highlighter-rouge">&lt;article&gt;</code> tag on your console. We’re almost there!</p>

<p>Now, we don’t want the full article, only its headline and summary. Looking closer at the HTML we get, we see an <code class="language-plaintext highlighter-rouge">h2</code> and the first <code class="language-plaintext highlighter-rouge">p</code> that look promising. Let’s refactor our code a bit to have <code class="language-plaintext highlighter-rouge">firstArticle</code> as a variable we can use, create a function to be used for both the header and the summary, and pluck both of them to show on the console:</p>

<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code>     'https://nytimes.com/section/todayspaper'
   );
 
<span class="gd">-  const firstArticle = await page.$eval(
-    'article:first-of-type',
-    e =&gt; e.outerHTML
-  );
</span><span class="gi">+  const firstArticle = await page.$('article:first-of-type');
+
+  const getText = (parent, selector) =&gt; {
+    return parent.$eval(selector, el =&gt; el.innerText);
+  };
+
+  const header = await getText(firstArticle, 'h2');
+  const summary = await getText(firstArticle, 'p:first-of-type');
</span> 
<span class="gd">-  console.log(firstArticle);
</span><span class="gi">+  console.log(`${header}\n${summary}`);
</span> 
   await browser.close();
 })();
</code></pre></div></div>

<p>Go ahead, run that on the terminal and you show see two lines, the top on as the header and the bottom one as the summary of the article!</p>

<p>To be honest, that’s it! 🎉 <strong>A web scraper doesn’t need to be fancy or complicated</strong>, it really depends on what you are trying to fetch from a page. I had one running for a few days a while back (which I’ll write about on a following article) and it was basically doing thigs on another page, just checking if a specific string of text has changed already or not.</p>

<p>Having said that, there is <em>so much more</em> you can do with Puppeteer — the sky is the limit. Check <a href="https://devdocs.io/puppeteer/">their documentation</a> to see the available methods, <a href="https://github.com/puppeteer/examples">official examples</a> of wild things you can use it for, and you can even use it to <a href="https://addyosmani.com/blog/puppeteer-recipes/">automate performance work</a>!</p>

<p>See you around soon for the second part of this article…</p>]]></content><author><name></name></author><category term="javascript" /><category term="puppeteer" /><summary type="html"><![CDATA[This is the first post of a 3-part project that will show you how to build a web scraper, add notifications, deploy it and make sure it runs regularly. This is the series:]]></summary></entry><entry><title type="html">Launching ‘Heroes of Computer Science’</title><link href="http://blog.gnclmorais.com/heroes-of-computer-science" rel="alternate" type="text/html" title="Launching ‘Heroes of Computer Science’" /><published>2021-05-10T00:00:00+00:00</published><updated>2021-05-10T00:00:00+00:00</updated><id>http://blog.gnclmorais.com/heroes-of-computer-science</id><content type="html" xml:base="http://blog.gnclmorais.com/heroes-of-computer-science"><![CDATA[<p>A few months ago, after learning about the death of <a href="https://en.wikipedia.org/wiki/Jean_E._Sammet">Jean E. Sammet</a>, I realised that there is this huge amount of personalities that pushed the field of Computer Science and that, even though I went through a master’s degree in Computer Engineering, I still didn’t know who most of them were…</p>

<p>This gave me an idea for a project. A short newsletter (with an RSS feed as well, for people with <em>email fatigue</em>) that, every fortnight or so, would introduce you to someone that (in some way) propelled Computer Science forward, someone you should know a little about.</p>

<p>This eventually led me to <a href="heroesofcomputer.science"><strong>Heroes of Computer Science</strong></a> and I’ve sent three issues so far: <a href="https://www.heroesofcomputer.science/issues/heroes-of-computer-science-issue-1-499264">Ada Lovelace</a>, <a href="https://www.heroesofcomputer.science/issues/heroes-of-computer-science-issue-2-567730">Grace Hopper</a> and <a href="https://www.heroesofcomputer.science/issues/heroes-of-computer-science-issue-3-578033">Alonzo Church</a>.</p>

<p>I have many more people to go through, but so far, I’m really enjoying discovering about new personalities and re-learning about names I already knew about but not that well.</p>

<p>If you have a suggestion for someone to be spotlighted on this project, let mw know through <a href="https://airtable.com/shrzC1ttxMN9cdXFI">this simple form</a>! I’d love to get more ideas for future issues. 🤩</p>

<p><img src="/images/posts/2021-05-10-HoCS.png" alt="Heroes of Computer Science banner" /></p>]]></content><author><name></name></author><category term="projects" /><summary type="html"><![CDATA[A few months ago, after learning about the death of Jean E. Sammet, I realised that there is this huge amount of personalities that pushed the field of Computer Science and that, even though I went through a master’s degree in Computer Engineering, I still didn’t know who most of them were…]]></summary></entry><entry><title type="html">Running bash commands with git alias</title><link href="http://blog.gnclmorais.com/running-bash-commands-with-git-alias" rel="alternate" type="text/html" title="Running bash commands with git alias" /><published>2021-05-08T14:00:00+00:00</published><updated>2021-05-08T14:00:00+00:00</updated><id>http://blog.gnclmorais.com/running-bash-commands-with-git-alias</id><content type="html" xml:base="http://blog.gnclmorais.com/running-bash-commands-with-git-alias"><![CDATA[<p>This is a quick tip I learned recently and it allowed me to improve my <code class="language-plaintext highlighter-rouge">git</code> workflow a bit more.</p>

<p>Here’s a new part of my <code class="language-plaintext highlighter-rouge">.gitconfig</code>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[alias]
    main = !git checkout main &amp;&amp; git pull --no-tags
    sync = !git main &amp;&amp; git switch -
    fuse = !git sync &amp;&amp; git rebase main
</code></pre></div></div>

<p>This week I added these three alias to my <code class="language-plaintext highlighter-rouge">.gitconfig</code> but, if you notice, they are not “regular” alias. <strong>The <code class="language-plaintext highlighter-rouge">!</code> at their beginning tells Git that these are not alias to Git commands, but rather bash commands.</strong></p>

<p>This gives you a simple but powerful way to chain executions, so I created three related alias that I can call depending on my goal:</p>
<ul>
  <li>If I finished working on a branch and I want to get back to <code class="language-plaintext highlighter-rouge">main</code> and start with the most recent codebase, I’ll run <code class="language-plaintext highlighter-rouge">git main</code>. Notice the <code class="language-plaintext highlighter-rouge">--no-tags</code>, this is motivated by working on a <em>large</em> monorepo and not needing all the tags of the packages we keep updating;</li>
  <li>If I’m working on a branch and I want to quickly get any changes made to our <code class="language-plaintext highlighter-rouge">main</code> branch but come back to the branch I’m in right now, I’ll call <code class="language-plaintext highlighter-rouge">git sync</code>. As a note, <code class="language-plaintext highlighter-rouge">git switch -</code> gets you back to the branch you were before you moved to the current branch you are now;</li>
  <li>Finally, if I want to bring the current branch I’m at up to speed with the latest code we’ve shipped, I’ll use <code class="language-plaintext highlighter-rouge">git fuse</code>. It will do everything I described on the other commands so far <em>and</em> rebase our <code class="language-plaintext highlighter-rouge">main</code> branch onto the current branch I’m at.</li>
</ul>]]></content><author><name></name></author><category term="git" /><category term="bash" /><summary type="html"><![CDATA[This is a quick tip I learned recently and it allowed me to improve my git workflow a bit more.]]></summary></entry><entry><title type="html">`this.something = ‘else’` vs `this.set(‘something’, ‘else’)` in Ember</title><link href="http://blog.gnclmorais.com/this-set-in-ember" rel="alternate" type="text/html" title="`this.something = ‘else’` vs `this.set(‘something’, ‘else’)` in Ember" /><published>2021-01-31T00:00:00+00:00</published><updated>2021-01-31T00:00:00+00:00</updated><id>http://blog.gnclmorais.com/this-set-in-ember</id><content type="html" xml:base="http://blog.gnclmorais.com/this-set-in-ember"><![CDATA[<p>I’ve been getting deeper into Ember.js in my new job, and I learn something new every day. Sometimes, things you’ve been using here and there are a bit mysterious. Eventually, either you or someone else notices them and raises insightful questions, like Michal asked on <a href="https://discord.com/channels/480462759797063690/480523424121356298/803933292444909610">Ember’s Discord channel</a>. The question was basically this:</p>

<blockquote>
  <p>What’s the difference between <code class="language-plaintext highlighter-rouge">this.something = 'else'</code> and <code class="language-plaintext highlighter-rouge">this.set('something', 'else')</code> in tests?</p>
</blockquote>

<p>The answer is surprisingly simple (at the surface, we don’t need to get deep into Ember’s reactivity): <code class="language-plaintext highlighter-rouge">this.set</code> is the way you have to trigger re-render in test. For example, if you set a value with <code class="language-plaintext highlighter-rouge">this.value = 'test'</code> and render a component that will use that <code class="language-plaintext highlighter-rouge">this.value</code>, changing the value of <code class="language-plaintext highlighter-rouge">this.value</code> now <strong>will not</strong> re-render the partial. If, instead, you set that value with <code class="language-plaintext highlighter-rouge">this.set('value', 'test')</code>, whenever you change it again with <code class="language-plaintext highlighter-rouge">this.set('value', 'something else')</code>, the template will automatically re-render.</p>

<p>Small caveat (like everything in like): re-rendering will only occur if the value you are reassigning is different that the value previously there, <a href="https://discord.com/channels/480462759797063690/480523424121356298/804027937397276690">as it seems</a>.</p>]]></content><author><name></name></author><category term="ember" /><summary type="html"><![CDATA[I’ve been getting deeper into Ember.js in my new job, and I learn something new every day. Sometimes, things you’ve been using here and there are a bit mysterious. Eventually, either you or someone else notices them and raises insightful questions, like Michal asked on Ember’s Discord channel. The question was basically this:]]></summary></entry><entry><title type="html">Advent of Code 2020 and Ruby</title><link href="http://blog.gnclmorais.com/advent-of-code-2020-and-ruby" rel="alternate" type="text/html" title="Advent of Code 2020 and Ruby" /><published>2020-12-15T00:00:00+00:00</published><updated>2020-12-15T00:00:00+00:00</updated><id>http://blog.gnclmorais.com/advent-of-code-2020-and-ruby</id><content type="html" xml:base="http://blog.gnclmorais.com/advent-of-code-2020-and-ruby"><![CDATA[<p>After a year filled with so much <a href="https://www.merriam-webster.com/dictionary/sameness"><em>sameness</em></a> and spending way too much time at home for our own sake (and others), I’m surprised that tackling the daily challenges of <a href="https://adventofcode.com/2020/">Advent of Code</a> have been keeping me happy and entertained.</p>

<p>As a break from my daily work with JavaScript and to keep my skills sharp, I’ve picked Ruby for it. Its performance shouldn’t be a problem for this kind of brain teasers and its syntactic sugar is always welcomed.</p>

<p>While trying to solve the second part of <a href="https://adventofcode.com/2020/day/14">day 14</a>, I found myself hogging my computer over and over again, trying to validate my algorithm. I was forced to fire up Activity Monitor and kill one of my <code class="language-plaintext highlighter-rouge">ruby</code> processes over and over again, which would be consuming several gigabytes of RAM! What was happening??</p>

<p>Even after finding a possible solution I was happy with, this kept happening — which meant I couldn’t get the answer to the puzzle. After stopping, reflecting a bit, and probing a few parts of the code, I had an idea — and it worked!</p>

<p>The challenge is about reading an input file and, with the values you get, write certain numbers in memory so you can sum them all up in the end. In my case, I was using an array to emulate the program’s memory and its indexes as the memory addresses.</p>

<p>This worked out well for a while… until I tried to write a value to the index <code class="language-plaintext highlighter-rouge">14694662143</code>! 💥 <strong>This</strong> was the culprit and, while I can’t find the maximum number of an array’s index in Ruby, I found a <a href="https://stackoverflow.com/a/6865199">few places</a> that seem to point out that I was <em>way</em> above the realm of possibility.</p>

<p>The fix was actually quite simple:</p>
<ul>
  <li>instead of using an array (<code class="language-plaintext highlighter-rouge">[]</code>) to keep the addresses and <code class="language-plaintext highlighter-rouge">arr.compact.reduce(:+)</code> to sum all the numbers hold,</li>
  <li>I just changed it to a hash (<code class="language-plaintext highlighter-rouge">{}</code>) which still allows me to keep the values and use <code class="language-plaintext highlighter-rouge">arr.values.reduce(:+)</code> to sum them.</li>
</ul>

<p>I guess Ruby tries to allocate an array as big as your index whenever you do something like this. If the index is <em>quite big</em>, Ruby will obey you but create some headaches. A hash won’t, you’ll only allocate what you assign, nothing in between.</p>]]></content><author><name></name></author><category term="advent-of-code" /><category term="ruby" /><summary type="html"><![CDATA[After a year filled with so much sameness and spending way too much time at home for our own sake (and others), I’m surprised that tackling the daily challenges of Advent of Code have been keeping me happy and entertained.]]></summary></entry><entry><title type="html">DigitalOcean and Dokku to replace Heroku</title><link href="http://blog.gnclmorais.com/digitalocean-dokku-to-replace-heroku" rel="alternate" type="text/html" title="DigitalOcean and Dokku to replace Heroku" /><published>2020-10-21T00:00:00+00:00</published><updated>2020-10-21T00:00:00+00:00</updated><id>http://blog.gnclmorais.com/digitalocean-dokku-to-replace-heroku</id><content type="html" xml:base="http://blog.gnclmorais.com/digitalocean-dokku-to-replace-heroku"><![CDATA[<p>I’ve had this on my to-do list for ages but I’ve only recently looked into it. <a href="https://heroku.com">Heroku</a> is a great tool to quickly deploy and set up projects but you start accumulating a hefty bill if you start moving your work out of their free plans. $7 a dyno (without counting any addons you might need, like databases) becomes unmanageable if you want to spin up multiple small projects with small traffic but without downtime (remember, Heroku only provides you a certain amount of <a href="https://devcenter.heroku.com/articles/free-dyno-hours#dyno-sleeping">free hours</a> per month).</p>

<p>To go around this, I finally started playing around with <a href="https://github.com/dokku/dokku#dokku">Dokku</a> on <a href="https://m.do.co/c/56af94308a04">DigitalOcean</a>. There is even an app available <a href="https://marketplace.digitalocean.com/apps/dokku">straight from their platform</a>, so the support seems to be solid. The premise is simple: create a $5 droplet on DigitalOcean with Dokku installed on it (which you can do straight from <a href="https://marketplace.digitalocean.com/apps/dokku">here</a>) and you’ll be able to use it as your personal Heroku.</p>

<p>The deployment process is quite similar. Here’s a sample output of what I get when I push a Ruby on Rails app to my own Dokku droplet (a few details removed/changed for privacy reasons):</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ git push dokku master

Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 309 bytes | 309.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
-----&gt; Cleaning up...
-----&gt; Building dokku-test-app from herokuish...
-----&gt; Adding BUILD_ENV to build environment...
-----&gt; Warning: Multiple default buildpacks reported the ability to handle this app. The first buildpack in the list below will be used.
       Detected buildpacks: multi ruby nodejs
-----&gt; Multipack app detected
=====&gt; Downloading Buildpack: https://github.com/heroku/heroku-buildpack-ruby.git
=====&gt; Detected Framework: Ruby
-----&gt; Installing bundler 2.1.4
-----&gt; Removing BUNDLED WITH version in the Gemfile.lock
-----&gt; Compiling Ruby/Rails
-----&gt; Using Ruby version: ruby-2.5.8
-----&gt; Installing dependencies using bundler 2.1.4
       Running: BUNDLE_WITHOUT='development:test' BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1 bundle install -j4
       Using rake 12.3.2
       …
       Using webpacker 3.6.0
       Bundle complete! 29 Gemfile dependencies, 74 gems now installed.
       Gems in the groups development and test were not installed.
       Bundled gems are installed into `./vendor/bundle`
       Bundle completed (0.79s)
       Cleaning up the bundler cache.
-----&gt; Installing node-v12.16.2-linux-x64
-----&gt; Installing yarn-v1.22.4
-----&gt; Detecting rake tasks
-----&gt; Preparing app for Rails asset pipeline
       Running: rake assets:precompile
       yarn install v1.22.4
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       [3/4] Linking dependencies...
       [4/4] Building fresh packages...
       Done in 87.94s.
       Webpacker is installed 🎉 🍰
       Using /tmp/build/config/webpacker.yml file for setting up webpack paths
       Compiling…
       Compiled all packs in /tmp/build/public/packs
       Asset precompilation completed (308.00s)
       Cleaning assets
       Running: rake assets:clean
-----&gt; Detecting rails configuration

=====&gt; Downloading Buildpack: https://github.com/heroku/heroku-buildpack-nodejs
=====&gt; Detected Framework: Node.js

-----&gt; Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error
       USE_YARN_CACHE=true
       NODE_ENV=production
       NODE_MODULES_CACHE=true
       NODE_VERBOSE=false

-----&gt; Installing binaries
       engines.node (package.json):  unspecified
       engines.npm (package.json):   unspecified (use default)
       engines.yarn (package.json):  unspecified (use default)

       Resolving node version 12.x...
       Downloading and installing node 12.19.0...
       Using default npm version: 6.14.8
       Resolving yarn version 1.22.x...
       Downloading and installing yarn (1.22.10)
       Installed yarn 1.22.10
       !     node_modules checked into source control
       https://devcenter.heroku.com/articles/node-best-practices#only-git-the-important-bits


-----&gt; Restoring cache
       - yarn cache

-----&gt; Installing dependencies
       Installing node modules (yarn.lock)
       yarn install v1.22.10
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       [3/4] Linking dependencies...
       [4/4] Building fresh packages...
       Done in 45.75s.

-----&gt; Build

-----&gt; Pruning devDependencies
       yarn install v1.22.10
       [1/4] Resolving packages...
       [2/4] Fetching packages...
       [3/4] Linking dependencies...
       [4/4] Building fresh packages...
       Done in 16.75s.

-----&gt; Caching build
       - yarn cache

-----&gt; Build succeeded!
       !     Unmet dependencies don't fail yarn install but may cause runtime issues
       https://github.com/npm/npm/issues/7494

       Using release configuration from last framework (Node.js).
-----&gt; Discovering process types
       Procfile declares types -&gt; release, web
-----&gt; Releasing dokku-test-app...
-----&gt; Deploying dokku-test-app...
 !     Release command declared: 'rake db:migrate 2&gt;/dev/null || rake db:setup'
       D, [2020-10-24T18:04:32.244136 #13] DEBUG -- :    (2.0ms)  SELECT pg_try_advisory_lock(7827836239135902150)
       D, [2020-10-24T18:04:32.272390 #13] DEBUG -- :    (2.9ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
       D, [2020-10-24T18:04:32.281930 #13] DEBUG -- :   ActiveRecord::InternalMetadata Load (0.8ms)  SELECT  "ar_internal_metadata".* FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = $1 LIMIT $2  [["key", "environment"], ["LIMIT", 1]]
       D, [2020-10-24T18:04:32.290196 #13] DEBUG -- :    (0.3ms)  BEGIN
       D, [2020-10-24T18:04:32.291934 #13] DEBUG -- :    (0.3ms)  COMMIT
       D, [2020-10-24T18:04:32.292759 #13] DEBUG -- :    (0.4ms)  SELECT pg_advisory_unlock(7827836239135902150)
-----&gt; App Procfile file found
       DOKKU_SCALE declares scale -&gt; release=0 web=1
=====&gt; Processing deployment checks
       No CHECKS file found. Simple container checks will be performed.
       For more efficient zero downtime deployments, create a CHECKS file. See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples
-----&gt; Attempting pre-flight checks (web.1)
       Waiting for 10 seconds ...
       Default container check successful!
-----&gt; Running post-deploy
-----&gt; Configuring dokku-test-app.Dokku...(using built-in template)
-----&gt; Creating http nginx.conf
       Reloading nginx
-----&gt; Renaming containers
       Found previous container(s) (11c86d03fec9) named dokku-test-app.web.1
       Renaming container (11c86d03fec9) dokku-test-app.web.1 to dokku-test-app.web.1.1603562703
       Renaming container (154edf6916aa) inspiring_mayer to dokku-test-app.web.1
-----&gt; Shutting down old containers in 60 seconds
       11c86d03fec92a9f08e4d3c1dfa33ffad428d621ed951195684ab5255a529f19
=====&gt; Application deployed:
       http://dokku-test-app.Dokku

To 151.242.42.151:dokku-test-app
   92bf7df..a5b5166  master -&gt; master
</code></pre></div></div>

<p>P.S.: If you are interested in trying out DigitalOcean, you can use <a href="https://m.do.co/c/56af94308a04">this referral link</a> and get $100 straight into your account to spin up some droplets and try it yourself.</p>

<h2 id="troubleshooting">Troubleshooting</h2>

<p>Since everyone runs into a few hiccups when we’re trying something new, I’m going to keep this section as a living document of issues I’ve ran into while playing around with Dokku. Whenever I find a new issue that I’ve solved, I’ll come back to this blog post and update it.</p>

<h3 id="virtual-memory-exhausted-cannot-allocate-memory">virtual memory exhausted: Cannot allocate memory</h3>

<p>This one took me a while to find a solution for, but I believe that was just because I was not using the right terms. The <a href="http://dokku.viewdocs.io/dokku~v0.11.0/getting-started/advanced-installation/#vms-with-less-than-1gb-of-memory">documentation even has a solution</a> for it, it just took me a while to notice it. While installing the <code class="language-plaintext highlighter-rouge">sassc</code> gem, my droplet was running out of memory. This seems to happen because DigitalOcean’s basic droplets only have 512MB available, which might not be enough in a few situations. The documentation provides some instructions to follow that solved my issue, by creating a bigger <a href="https://en.wikipedia.org/wiki/Paging#Implementations">swap file</a> (acting like extra RAM):</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> /var
<span class="nb">touch </span>swap.img
<span class="nb">chmod </span>600 swap.img

<span class="nb">dd </span><span class="k">if</span><span class="o">=</span>/dev/zero <span class="nv">of</span><span class="o">=</span>/var/swap.img <span class="nv">bs</span><span class="o">=</span>1024k <span class="nv">count</span><span class="o">=</span>1000
mkswap /var/swap.img
swapon /var/swap.img
free

<span class="nb">echo</span> <span class="s2">"/var/swap.img    none    swap    sw    0    0"</span> <span class="o">&gt;&gt;</span> /etc/fstab
</code></pre></div></div>

<h2 id="further-reading">Further reading</h2>

<ul>
  <li><a href="https://medium.com/@herowebdev1/the-ultimate-digitalocean-dokku-deploy-guide-rails-other-environemnts-d28110f3dc86">The Ultimate Digitalocean Dokku Deploy Guide (Rails &amp; other environemnts)</a></li>
  <li><a href="https://medium.com/@dpaluy/the-ultimate-guide-to-dokku-and-ruby-on-rails-5-9ecad2dba4a3">The Ultimate Guide to Dokku and Ruby On Rails 5</a></li>
</ul>]]></content><author><name></name></author><category term="heroku" /><category term="dokku" /><category term="deployment" /><summary type="html"><![CDATA[I’ve had this on my to-do list for ages but I’ve only recently looked into it. Heroku is a great tool to quickly deploy and set up projects but you start accumulating a hefty bill if you start moving your work out of their free plans. $7 a dyno (without counting any addons you might need, like databases) becomes unmanageable if you want to spin up multiple small projects with small traffic but without downtime (remember, Heroku only provides you a certain amount of free hours per month).]]></summary></entry></feed>