Jekyll2020-10-15T11:42:23+00:00https://jatindhankhar.in/feed.xmlWeblog of JDContains source code of my blogJatin DhankharResponsible Disclosure: Breaking out of a Sandboxed Editor to perform RCE2020-02-04T00:00:00+00:002020-02-04T00:00:00+00:00https://jatindhankhar.in/blog/responsible-disclosure-breaking-out-of-a-sandboxed-editor-to-perform-rce<h1 id="tldr">tl;dr</h1>
<p>Found a way to escape the sandboxed editor to perform Remote Code Execution which leads to the ability to view AWS credentials, SSL certificate and other stuff.</p>
<p>Pretty much owning the entire machine :smiling_imp:</p>
<h1 id="story---finding-the-issue">Story - Finding the issue</h1>
<p>If you are still here after reading the tl;dr, I guess you are here for the story?<br />
So, let me give you one.</p>
<p>While doing recon I found many sub-domains and IP addresses belonging to HackerEarth, one of them was <a href="https://\[READACTED\]/#/home/node/he-theia/sandbox">https://[REDACTED]/#/home/node/he-theia/sandbox</a> which was running an online ide built on top of vs-code named <a href="https://theia-ide.org/" title="https://theia-ide.org/">Theia IDE</a>.</p>
<p>At first glance, it looked pretty boring, after all, it’s an IDE running in a browser (wait, that’s normal since most of them are electron based :neutral_face: )</p>
<p>So, anyway, I played around with it for a while, the ultimate goal was to execute random code on the machine. But, they removed the terminal view command from the IDE shortcuts and menu. So, I tried to “run” the code file but that option was also not available.</p>
<p>Then poking around I tried “Task: Run selected text” by bringing up the global action menu shortcut from vscode (ctrl/cmd + shift + p) and lo, and behold, it opened up a terminal.
<img src="/images/disclosure-hackerearth/run_selected_text_prompt.png" /></p>
<p>One I got the terminal access, it was easy to demonstrate the RCE.</p>
<p><img src="/images/disclosure-hackerearth/terminal_prompt.png" /></p>
<h3 id="trying-out-things">Trying out things</h3>
<p>I tried with the possibility of reading system config files and was able to read HackearEarth’s private SSL <code class="language-plaintext highlighter-rouge">.crt</code> and <code class="language-plaintext highlighter-rouge">.key</code> files.
Pretty much, most of the system configuration files.</p>
<p><img src="/images/disclosure-hackerearth/ssl_certificates.png" /></p>
<p><img src="/images/disclosure-hackerearth/ssl_private_key.png" /></p>
<p>I was even able to read the git log and original <code class="language-plaintext highlighter-rouge">ide_fetcher.py</code> that powered the ide initial startup commands since the repo cloned still had git metadata.</p>
<p><img src="/images/disclosure-hackerearth/git_config.png" /></p>
<p><img src="/images/disclosure-hackerearth/git_log.png" /></p>
<p>Through some command-line fu, I was able to read the original arguments used to invoke the web-ide.</p>
<p><img src="/images/disclosure-hackerearth/command_line_arguments.png" /></p>
<h3 id="reading-aws-credentials">Reading AWS Credentials</h3>
<p>After it was clear that I was able to read system files, write arbitrary files and command, I wanted to see if it was possible to use the terminal to read AWS credentials since the instance was hosted on AWS infrastructure just like the rest of the HackerEarth’s infrastructure.</p>
<p>I first tried the usual metadata URL to access aws details</p>
<p><code class="language-plaintext highlighter-rouge">curl http://169.254.169.254/latest/api/token</code> but it didn’t work instead it gave me <code class="language-plaintext highlighter-rouge">curl: failed to connect</code>.</p>
<p>Lost, I tried to ping the domain that also didn’t work. Then I found a blog by Puma Scan on <a href="https://pumascan.com/resources/cloud-security-instance-metadata/">Cloud Security - Attacking The Metadata Service</a></p>
<p>There it was mentioned that attacking ECS metadata was different from attacking EC2 metadata service since it was served from a different domain. Then I checked the environment variable output again which I ignored earlier for some reason :sweat_smile: and it was right there in front of my eyes the whole time.</p>
<p><img src="/images/disclosure-hackerearth/env_output.png" /></p>
<p>It contained both the <code class="language-plaintext highlighter-rouge">ECS_CONTAINER_METADATA_URI</code> and <code class="language-plaintext highlighter-rouge">AWS_CONTAINER_CREDENTIALS_RELATIVE_URI</code></p>
<p>After that, it was just a <code class="language-plaintext highlighter-rouge">curl</code> away :smile:</p>
<p><img src="/images/disclosure-hackerearth/aws_metadata.png" /></p>
<p><img src="/images/disclosure-hackerearth/aws_creds.png" /></p>
<h1 id="timeline">Timeline</h1>
<ul>
<li>Tue, Dec 24 [ 7:35 PM IST] - Reached out to HackerEarth support about the issue</li>
<li>Wed, Dec 25 [ 9:50 AM IST] - HackerEarth support team asked to submit the issue to them so that they can forward it to security team (Although I wanted to report it directly to security team)</li>
</ul>
<p>< Back and forth regarding submission ></p>
<ul>
<li>Wed, Dec 25, 2019 [ 10:25 PM IST ] - Submitted the issue along with detailed POC and evidence</li>
<li>Thu, Dec 26, 2019 [ 05:49 PM IST ] - The instance was down. I asked HackerEarth for the confirmation of the fix.</li>
<li>Fri, Dec 27, 2019 [ 12:57 PM IST] - HackerEarth confirmed the fix from their end.</li>
<li>Mon, Jan 13, 2020 [12:32 PM IST] - Bounty Awarded</li>
<li>Thu, Jan 23, 2020 [4:21 PM IST] - Bounty received</li>
<li>Thu, Jan 23, 2020 [11:10 PM IST] - Disclosure draft shared</li>
<li>Tue, Feb 04, 2020 [06:03 PM IST] - Disclosure draft approved</li>
<li>Tue, Feb 04, 2020 [06:25 PM IST] - Blog published</li>
</ul>Jatin DhankharResponsible Disclosure: Breaking out of a Sandboxed Editor to perform RCEAuto Refresh tokens in Ruby using Procs2019-12-15T00:00:00+00:002019-12-15T00:00:00+00:00https://jatindhankhar.in/blog/auto-refresh-tokens-in-ruby-using-procs<h1 id="backstory-">Backstory ?</h1>
<p>Few months ago I had to integrate a third party service into our system.<br />
The third party in question was used in a dashboard to verify certain details of users.</p>
<p>So in an ideal world, the end user just clicks on verify and our system handles rest of the stuff (like it should).</p>
<p>So the service had an api which can be used to verify those details (Obviously :smile:) but it used a refresh token which expired every 15 minutes.</p>
<p><a href="https://auth0.com/learn/refresh-tokens/">Refresh tokens are good from security point of view</a> but using them in normal flow can be tricky sometimes since they are short lived and cannot be reused.</p>
<h1 id="how-">How ?</h1>
<p>We needed to handle the refresh of the token gracefully without impacting the original request.</p>
<p>The verification call was wrapped in a proc, so that it be can be passed around.</p>
<p>If the request returned unauthorised access, we could regenerate the token and retry the original request.</p>
<p>This is what I came up with</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="k">class</span> <span class="nc">APIClient</span>
<span class="k">class</span> <span class="nc">RefreshTokenError</span> <span class="o"><</span> <span class="no">Exception</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">msg</span> <span class="o">=</span> <span class="s2">"Failed to Refresh Token"</span><span class="p">)</span>
<span class="k">super</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">AUTH_ERROR_CODE</span> <span class="o">=</span> <span class="mi">400</span>
<span class="no">SERVER_ERROR_CODE</span> <span class="o">=</span> <span class="mi">520</span>
<span class="no">ERROR_STATUS</span> <span class="o">=</span> <span class="s2">"ERROR"</span>
<span class="no">BASE_URL</span> <span class="o">=</span> <span class="s2">"https://api-domain.tld"</span>
<span class="no">RETRY_THRESHOLD</span> <span class="o">=</span> <span class="mi">3</span>
<span class="no">CLIENT_ID</span> <span class="o">=</span> <span class="no">ENV</span><span class="p">[</span><span class="s1">'CLIENT-ID'</span><span class="p">]</span>
<span class="no">CLIENT_SECRET</span> <span class="o">=</span> <span class="no">ENV</span><span class="p">[</span><span class="s1">'CLIENT-SECRET'</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@token</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="vi">@retries</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">refresh_and_execute</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="nf">call</span>
<span class="k">return</span> <span class="n">response</span> <span class="k">if</span> <span class="n">auth_success?</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
<span class="k">while</span> <span class="vi">@retries</span> <span class="o"><</span> <span class="no">RETRY_THRESHOLD</span>
<span class="k">return</span> <span class="n">request</span><span class="p">.</span><span class="nf">call</span> <span class="k">if</span> <span class="n">refresh_token</span>
<span class="vi">@retries</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">raise</span> <span class="no">RefreshTokenError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">validate_details</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="n">request_call</span> <span class="o">=</span> <span class="no">Proc</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span><span class="no">HTTParty</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="n">validation_endpoint</span><span class="p">,</span> <span class="ss">query: </span><span class="n">query</span><span class="p">,</span> <span class="ss">headers: </span><span class="n">token_headers</span><span class="p">,</span> <span class="ss">timeout: </span><span class="mi">2</span><span class="p">)}</span>
<span class="n">refresh_and_execute</span><span class="p">(</span><span class="n">request_call</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">auth</span>
<span class="n">response</span> <span class="o">=</span> <span class="no">HTTParty</span><span class="p">.</span><span class="nf">post</span><span class="p">(</span><span class="n">authorization_endpoint</span><span class="p">,</span> <span class="ss">headers: </span><span class="n">auth_headers</span><span class="p">)</span>
<span class="n">token_updated</span> <span class="o">=</span> <span class="kp">false</span>
<span class="k">if</span> <span class="n">response</span><span class="p">.</span><span class="nf">success?</span>
<span class="k">unless</span> <span class="n">response</span><span class="p">.</span><span class="nf">parsed_response</span><span class="p">.</span><span class="nf">nil?</span>
<span class="vi">@token</span> <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="nf">parsed_response</span><span class="p">.</span><span class="nf">fetch</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="p">{}).</span><span class="nf">fetch</span><span class="p">(</span><span class="s2">"token"</span><span class="p">,</span> <span class="kp">nil</span><span class="p">)</span>
<span class="n">token_updated</span> <span class="o">=</span> <span class="o">!</span><span class="vi">@token</span><span class="p">.</span><span class="nf">nil?</span>
<span class="k">end</span>
<span class="vi">@retries</span> <span class="o">=</span> <span class="mi">0</span> <span class="k">if</span> <span class="n">token_updated</span>
<span class="k">end</span>
<span class="n">token_updated</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">refresh_token</span>
<span class="n">auth</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">authorization_endpoint</span>
<span class="no">URI</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="no">BASE_URL</span><span class="p">,</span> <span class="s2">"authorize_endpoint"</span><span class="p">).</span><span class="nf">to_s</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">validation_endpoint</span>
<span class="no">URI</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="no">BASE_URL</span><span class="p">,</span> <span class="s2">"verificationUrl"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">auth_headers</span>
<span class="p">{</span>
<span class="s1">'X-Client-Id'</span> <span class="o">=></span> <span class="no">CLIENT_ID</span><span class="p">,</span>
<span class="s1">'X-Client-Secret'</span> <span class="o">=></span> <span class="no">CLIENT_SECRET</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">token_headers</span>
<span class="p">{</span><span class="s1">'Authorization'</span> <span class="o">=></span> <span class="s2">"Bearer </span><span class="si">#{</span><span class="vi">@token</span><span class="si">}</span><span class="s2">"</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">auth_success?</span> <span class="n">response</span>
<span class="n">response</span><span class="p">.</span><span class="nf">success?</span> <span class="n">and</span> <span class="n">response</span><span class="p">.</span><span class="nf">parsed_response</span><span class="p">[</span><span class="s2">"status"</span><span class="p">]</span> <span class="o">!=</span> <span class="no">ERROR_STATUS</span> <span class="n">and</span> <span class="n">not</span> <span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">parsed_response</span><span class="p">[</span><span class="s2">"subCode"</span><span class="p">].</span><span class="nf">to_i</span><span class="p">.</span><span class="nf">between?</span><span class="p">(</span><span class="no">AUTH_ERROR_CODE</span><span class="p">,</span> <span class="no">SERVER_ERROR_CODE</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and using it like this</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">APIClient</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">validate_details</span><span class="p">(</span><span class="n">query</span><span class="p">).</span><span class="nf">parsed_response</span>
</code></pre></div></div>
<p>The magic (or just plain logic :sweat_smile: ) happens in <code class="language-plaintext highlighter-rouge">refresh_and_execute</code></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="k">def</span> <span class="nf">refresh_and_execute</span><span class="p">(</span><span class="n">request</span><span class="p">)</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="nf">call</span>
<span class="k">return</span> <span class="n">response</span> <span class="k">if</span> <span class="n">auth_success?</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
<span class="k">while</span> <span class="vi">@retries</span> <span class="o"><</span> <span class="no">RETRY_THRESHOLD</span>
<span class="k">return</span> <span class="n">request</span><span class="p">.</span><span class="nf">call</span> <span class="k">if</span> <span class="n">refresh_token</span>
<span class="vi">@retries</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">raise</span> <span class="no">RefreshTokenError</span>
<span class="k">end</span>
</code></pre></div></div>
<p>It returns the original response if there was no authentication error.</p>
<p>Otherwise it attempts to refresh the token and retry the original request, until retry threshold is reached.</p>
<p><em>Is there a better to do it ? I would love to know more and improve the implementation.</em></p>Jatin DhankharAuto refresh API tokens seamlessly in ruby using procs.Migrating an active repo from Bitbucket to Github2019-12-01T00:00:00+00:002019-12-01T00:00:00+00:00https://jatindhankhar.in/blog/migrating-an-active-repo-from-bitbucket-to-github<h1 id="what">What?</h1>
<p>We needed to migrate a high commit frequency repository that was hosted on Bitbucket to Github.</p>
<p>The repository in question is a tier 1 service powering a critical part of the infrastructure so we needed to minimize the downtime (oh, throw in more fancy words, to make it sound important :satisfied: ) and make sure deployments were not blocked during the migration</p>
<h1 id="why">Why?</h1>
<p>We were using Bitbucket to host the code and it worked fine for basic use cases but we needed to support automated PR reviews to assist the code reviews and Bitbucket wasn’t really playing nice.<br />
There were many features that were available on Github but not on Bitbucket</p>
<p>Some of them were</p>
<ol>
<li><a href="https://github.blog/2019-02-14-introducing-draft-pull-requests/">Draft pull requests</a></li>
<li><a href="https://help.github.com/en/articles/about-pull-request-reviews">Explicit reviewer request changes</a></li>
<li><a href="https://help.github.com/en/articles/reviewing-proposed-changes-in-a-pull-request#marking-a-file-as-viewed">Collapsing files and marking files as already viewed</a></li>
</ol>
<p>Also, let’s just say, Bitbucket’s UI wasn’t pleasing to many. (<em>cough</em> No native syntax highlighting, and no way to hide comments <em>cough</em>)</p>
<p>We are using GoCD as CI pipeline along with <a href="https://github.com/prontolabs/pronto" title="Pronto">Pronto</a> and it also didn’t support Bitbucket’s API properly. So we listed down the reasons why we needed to migrate to Github.</p>
<p>Plus the rest of the organisation was already on Github, so it wasn’t that hard to convince people. Overall it was much easier to integrate new stuff with Github’s API than to deal with the Bitbucket’s API.</p>
<h1 id="how">How?</h1>
<h2 id="plan">Plan</h2>
<p>The first step was to clone the repo from Bitbucket to Github and do periodic syncs.<br />
During this whole process, we turned off branch protection for the holy branches (in our case it is <code class="language-plaintext highlighter-rouge">master</code> and <code class="language-plaintext highlighter-rouge">release</code>)<br />
Once a clone of the code was made, we then tested our pipelines for (package building and deployments) on one of the staging environments.<br />
After that it was just syncing the new repository periodically with new commits and once we were confident that it will work.<br />
Writes were disabled to the old repository.</p>
<p>Branch protection was turned on for the new repository.</p>
<p>Asked developers to use the new repository.</p>
<h2 id="execution">Execution</h2>
<p><strong>First clone</strong>
We followed this guide from Gist to migrate repo <a href="https://github.com/aiidateam/aiida-core/wiki/How-to-migrate-from-BitBucket-to-GitHub" title="https://github.com/aiidateam/aiida-core/wiki/How-to-migrate-from-BitBucket-to-GitHub">Github Migration guide</a></p>
<ol>
<li>Create the new repo on Github, preferably empty without any files, repo with starter files/templates will also work</li>
<li>Take a mirror clone of the repo from Bitbucket</li>
<li>Push the repo to the newly created Github repo</li>
</ol>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone <span class="nt">--mirror</span> https://bitbucket.org/username/old_repo.git
<span class="nb">cd </span>old_repo
git remote set-url <span class="nt">--push</span> origin git@github.com:username/new_repo
git push <span class="nt">--mirror</span>
</code></pre></div></div>
<p><strong>Periodic syncs</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd </span>old_repo
<span class="c"># Set remote if not already done git remote set-url --push origin git@github.com:username/new_repo</span>
<span class="c"># This pulls update from old repo </span>
git pull
<span class="c"># This pushes the sync from old repo to new one </span>
git push <span class="nt">--mirror</span>
</code></pre></div></div>Jatin DhankharMigrating an active repo with continuous pushes from Bitbucket to GithubHeisenbug: A bug that happened in background only2019-08-04T00:00:00+00:002019-08-04T00:00:00+00:00https://jatindhankhar.in/blog/heisenberg-bug-a-bug-that-happened-in-background-only<h1 id="the-bug">The bug</h1>
<p>Few months back I <del>discovered</del> encountered a <a href="https://en.wikipedia.org/wiki/Heisenbug" title="heisenbug">Heisenbug</a> (software bug that seems to disappear or alter its behaviour when one attempts to study it.)</p>
<h3 id="how-it-all-started-">How it all started ?</h3>
<p>We were about to release a new settlement process for a set of users, one feature of the settlement process was keeping old weeks settled. So, even if any ledger was added to a settled week, system would automatically counter it to maintain the original amount for that week.</p>
<p>We tested it thoroughly and were about to release it next day post final desk check.</p>
<p>I decided to test one more condition to be really sure and that’s when I encountered the heisenbug.</p>
<p>I tried to balance the amount of a user for a settlement week.</p>
<p>I did so by adding two ledgers for a settled week but for the same day.</p>
<p>I noticed that the processed week’s amount is not correct and found counters ledgers are not correct.</p>
<p>In place of one counter ledger of X amount, system made two counter ledgers of X amount !</p>
<p>Thinking that I must have made some changes, I reset the development branch to last good commit, test the normal scenario on rails console and it worked, system balanced the amount.</p>
<p>I ran it again via the original flow, system failed to balance it.</p>
<p><img src="https://en.meming.world/images/en/2/2c/Surprised_Pikachu_HD.jpg" alt="" /></p>
<p>Trying to find the issue, I added debug logs, fired up the console, triggered the function that did all of the work, it worked just fine !</p>
<p>I made the script, ran it through rails runner mode, it worked again.</p>
<p>But when I used the main flow it didn’t work. <strong>Main flow was triggered inside a Sidekiq Worker</strong></p>
<p>Struggling to find the bug, I opened a tmux window with split planes. Enabled activerecord query logs on both.</p>
<p>One with same flow in console and another containing sidekiq logs and stared at them.</p>
<p>Few minutes ago, Sajal from android team asked if rails does funny thing with threads and caching and I said don’t be absurd, rails can’t do this.</p>
<p>But then looking at the logs, it hit me.</p>
<p>Rails is indeed doing funny thing with sidekiq processes. Subsequent queries inside sidekiq was being cached.</p>
<p><img src="/images/cache_issue.png" alt="" /></p>
<h1 id="whyhow-it-happened-">Why/How it happened ?</h1>
<p>Console flow had following query</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># System computes difference
(2.1ms) SELECT SUM from table where date = 'DATE'
# System balances first entry
(2.1ms) SELECT SUM from table where date = 'DATE'
# System balances second entry
</code></pre></div></div>
<p>while sidekiq flow only had one real query and subsequent queries were cached</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># System computes difference
(2.1ms) SELECT SUM from table where date = 'DATE'
# System balances first entry
# System computes difference
CACHE (0ms) SELECT SUM from table where date = 'DATE'
# System balances second entry
</code></pre></div></div>
<p>After balancing first entry, system computed the difference after adding second query but since query was same, rails didn’t even fire the query, even though amount was affected and returned stale data and system tried to balance amount based on the stale data and reached in-consistent state.</p>
<p>We were trying to do aggregation for the same week and rails provided the stale query, since we performing the same query in rapid succession. Ideally rails clears cache on any write but it didn’t do it in this case, for some reason.</p>
<p>Now I knew what was happening I fixed the problem but let’s find out how it happened.</p>
<p>Quick Googling pointed to the <a href="https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting#activerecord-query-cache">Sidekiq Wiki</a> where it said</p>
<blockquote>
<p>Even when performing batched reads correctly, as above, the ActiveRecord query cache can cause memory bloat by storing query result</p>
<p>sets unnecessarily. Since Rails 5.0, the query cache is enabled by default for background jobs, including Sidekiq workers. If your job performs a large number of batched reads and is still using lots of memory, try <a href="http://api.rubyonrails.org/classes/ActiveRecord/QueryCache/ClassMethods.html">disabling the query cache</a>:</p>
</blockquote>
<p>ActiveRecord functionality was added to Sidekiq by this PR
<a href="https://github.com/mperham/sidekiq/pull/3718/files">https://github.com/mperham/sidekiq/pull/3718/files</a></p>
<p><strong>So if we are using Rails 5.x, queries in Sidekiq are cached by default</strong></p>
<h1 id="fixing-the-bug">Fixing the bug</h1>
<p>Fixing the bug was simple, we don’t want rails to cache the queries</p>
<p>So we can either.</p>
<ol>
<li>Clear the cache before every query ( Not good)</li>
<li>Disable caching for the method (Good)</li>
</ol>
<p>We can either wrap the entry block of sidekiq inside an <code class="language-plaintext highlighter-rouge">uncached</code> scope</p>
<p>like this</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">class</span> <span class="nc">Worker</span>
<span class="kp">include</span> <span class="no">Sidekiq</span><span class="o">::</span><span class="no">Worker</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">perform</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">uncached</span> <span class="k">do</span>
<span class="c1"># do stuff</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>or just wrap the function where calculation is performed. This ensures that it is never cached, no matter if it’s executed from console or from Sidekiq</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="nf">calculate</span>
<span class="no">Model</span><span class="p">.</span><span class="nf">uncached</span> <span class="k">do</span>
<span class="no">Model</span><span class="p">.</span><span class="nf">where</span><span class="p">(</span><span class="ss">date: </span><span class="no">Range</span><span class="p">).</span><span class="nf">sum</span><span class="p">(</span><span class="ss">:amount</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>Jatin DhankharDebugging an issue that occurred in background processes while running same query in rapid successionUsing Google App Script to Automate Treat Counter2019-07-27T00:00:00+00:002019-07-27T00:00:00+00:00https://jatindhankhar.in/blog/using-google-app-script-to-automate-treat-counter<h2 id="problem">Problem</h2>
<p>At work, my team follows <a href="https://www.scrum-institute.org/Daily_Scrum_Meeting.php" title="daily standup meetings">daily standup meeting</a> (tries to :sweat_smile: ). To ensure that people don’t miss standup, we maintain a ice cream counter (which is now a generic treat counter)</p>
<p><strong><em>Rule</em></strong></p>
<p>So if someone misses 4 standups, their counter increases by 1 and they need to treat the entire team to decrement the counter.</p>
<p>Treat counter is maintained as a google sheet and contain stats of every team member including their name, counter and other data.</p>
<p>Sheet is updated manually by treat committee (this can be automated as well )</p>
<p><img src="/images/Screenshot 2019-07-27 at 9.56.27 PM.png" alt="" /></p>
<p><strong><em>Issue</em></strong></p>
<p>It’s possible that people lose track of counter and someone needs to remind the person with pending treats.</p>
<p>Even though everyone secretly wants the treat, no one wants to be the one to remind the person with pending treats about the same :laughing: .</p>
<p><strong><em>Plan</em></strong></p>
<p>Secretly remind everyone about the treat and maintain anonymity, automate the process.</p>
<h2 id="solutions">Solutions</h2>
<p>There are many ways to automate this problem</p>
<ol>
<li>
<p><strong>Daily Cron:</strong> We can use a daily cron to pull the data from sheet using <a href="https://developers.google.com/sheets/api/">Google Sheets API</a> , parse the data then post it on the group using DingTalk’s Robot API.</p>
<p>Since sheet is updated manually, we don’t need to post the result everyday but only when someone’s treat crosses a certain threshold.</p>
</li>
<li>
<p><strong>Reactive Response (kind of):</strong> Since sheet is not updated daily and we need results to be posted only when someone updates the treats pending column, a reactive approach in response to update seems like a step in the right direction</p>
<p>We could have used Google app scripts update trigger but this was also a lot of work and involved handling sudden bursts and rollback of accidental counter upgrades.</p>
</li>
<li>
<p><strong>Using Google Sheet as a web app:</strong></p>
<p>I decided to go with this approach and tried to use sheet as a quick web app by using utilizing google sheets custom triggers.</p>
</li>
</ol>
<p>Now there is a big button inside the sheet itself, treat committee can update the counters and click on the button to announce the treat counter leader board.</p>
<p><strong><em>Google App Script</em></strong></p>
<p><a href="https://developers.google.com/apps-script/">Google app script is a super set of javascript</a>, (ES2015 to be precise ?), so I couldn’t use ES6 goodies and syntax sugar.</p>
<p>So when green button is clicked. Script does the following</p>
<ol>
<li>Fetches sheet specified by the <code class="language-plaintext highlighter-rouge">SHEET_ID</code> (usually part of the url itself)</li>
<li>Then converts each row and converts it to a object.</li>
<li>Filters people having more than 4 missed counts</li>
<li>Sorts them based on their treat pending counter</li>
<li>Posts the result by making a POST call to DingTalk’s robot api.</li>
</ol>
<p><img src="/images/Screenshot 2019-07-27 at 9.50.48 PM.png" alt="" /></p>
<p>So this is what I came up with, (code is not organized, this is not how I usually write code :see_no_evil:)</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">postTreatData</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">sheet</span> <span class="o">=</span> <span class="nx">SpreadsheetApp</span><span class="p">.</span><span class="nx">openById</span><span class="p">(</span><span class="dl">"</span><span class="s2">SHEET_ID</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">sheet</span><span class="p">.</span><span class="nx">getDataRange</span><span class="p">().</span><span class="nx">getValues</span><span class="p">();</span>
<span class="c1">// Start from second entry, since first entry is the columns itself</span>
<span class="kd">var</span> <span class="nx">memberStats</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">data</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">memberStats</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">getMemberStats</span><span class="p">(</span><span class="nx">data</span><span class="p">[</span><span class="nx">i</span><span class="p">]));</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">pending</span> <span class="o">=</span> <span class="nx">memberStats</span><span class="p">.</span><span class="nx">filter</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">stats</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">stats</span><span class="p">[</span><span class="dl">'</span><span class="s1">missed_count</span><span class="dl">'</span><span class="p">]</span> <span class="o">>=</span> <span class="mi">4</span><span class="p">;</span>
<span class="p">});</span>
<span class="nx">pending</span><span class="p">.</span><span class="nx">sort</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">a</span><span class="p">[</span><span class="dl">'</span><span class="s1">treats_pending</span><span class="dl">'</span><span class="p">]</span> <span class="o">></span> <span class="nx">b</span><span class="p">[</span><span class="dl">'</span><span class="s1">treats_pending</span><span class="dl">'</span><span class="p">];</span>
<span class="p">});</span>
<span class="nx">pending_messages</span> <span class="o">=</span> <span class="nx">pending</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">messageToMarkDownEntry</span><span class="p">);</span>
<span class="nx">postOnGroup</span><span class="p">(</span><span class="nx">pending_messages</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">));</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">getMemberStats</span><span class="p">(</span><span class="nx">rowData</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span> <span class="na">name</span><span class="p">:</span> <span class="nx">rowData</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="na">missed_count</span><span class="p">:</span> <span class="nb">parseInt</span><span class="p">(</span><span class="nx">rowData</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="mi">10</span><span class="p">),</span> <span class="na">given_count</span><span class="p">:</span> <span class="nb">parseInt</span><span class="p">(</span><span class="nx">rowData</span><span class="p">[</span><span class="mi">2</span><span class="p">],</span> <span class="mi">10</span><span class="p">),</span> <span class="na">treats_pending</span><span class="p">:</span> <span class="nb">parseInt</span><span class="p">(</span><span class="nx">rowData</span><span class="p">[</span><span class="mi">3</span><span class="p">],</span> <span class="mi">10</span><span class="p">)</span> <span class="p">};</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">messageToMarkDownEntry</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="dl">'</span><span class="s1">#### </span><span class="dl">'</span> <span class="o">+</span> <span class="nx">data</span><span class="p">[</span><span class="dl">'</span><span class="s1">name</span><span class="dl">'</span><span class="p">]</span> <span class="o">+</span> <span class="dl">"</span><span class="s2"> -- Treats Pending </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">data</span><span class="p">[</span><span class="dl">'</span><span class="s1">treats_pending</span><span class="dl">'</span><span class="p">];</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">postOnGroup</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">heading</span> <span class="o">=</span> <span class="dl">'</span><span class="s1"># 🍕🍔🍟 Treat LeaderBoard 🍧🍨🍰 </span><span class="se">\n</span><span class="s1"> # 🥄🍴 Be Ready ️🥣🥡🥢</span><span class="se">\n\n\n\n</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">footer</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">> ## [Treat Counter Sheet](https://docs.google.com/spreadsheets/d/SHEET_ID/) </span><span class="se">\n</span><span class="s1"> </span><span class="dl">'</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">formData</span> <span class="o">=</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">msgtype</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">markdown</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">markdown</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">title</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Ice Cream LeaderBoard</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">text</span><span class="dl">"</span><span class="p">:</span> <span class="p">[</span><span class="nx">heading</span><span class="p">,</span> <span class="nx">message</span><span class="p">,</span> <span class="nx">footer</span><span class="p">].</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">),</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">var</span> <span class="nx">headers</span> <span class="o">=</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">Accept</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">application/json</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">Content-Type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">application/json</span><span class="dl">"</span><span class="p">,</span>
<span class="p">};</span>
<span class="kd">var</span> <span class="nx">options</span> <span class="o">=</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">method</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">POST</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">headers</span><span class="dl">"</span><span class="p">:</span> <span class="nx">headers</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">payload</span><span class="dl">"</span><span class="p">:</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">formData</span><span class="p">)</span>
<span class="p">};</span>
<span class="nx">url</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">https://oapi.dingtalk.com/robot/send?access_token=YOUR_TOKEN</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">UrlFetchApp</span><span class="p">.</span><span class="nx">fetch</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">options</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Since DingTalk supports only a subset of markdown, I combined several string in markdown to form one big markdown document. If google app script supported ES6, I could have used string interpolation to make it more elegant.</p>
<p>On the ending note, after writing this one, I am missing more standups than I was attending early :p</p>Jatin DhankharHow I used google sheets with google app script to make a simple web app to automate treat counterCustom Http Header and Ruby Standard Library2019-07-20T00:00:00+00:002019-07-20T00:00:00+00:00https://jatindhankhar.in/blog/custom-http-header-and-ruby-standard-library<h1 id="the-problem">The problem</h1>
<p>One day at work I got an escalation from one of the third-party vendors that all the api calls to them were silently getting rejected on their end. They provided the explanation that one of the HTTP header that was used to supplying the api key itself, (let’s call it <code class="language-plaintext highlighter-rouge">API-KEY</code>) was being sent incorrectly.</p>
<p>They wanted the header to be a lower case like <code class="language-plaintext highlighter-rouge">api-key</code> but we started sending it in upper case <code class="language-plaintext highlighter-rouge">API-KEY</code>. This should not happen since we had a patch in place, to handle this situation.</p>
<h2 id="little-background-about-http-headers-and-nethttp">Little background about HTTP headers and NET::HTTP</h2>
<p>As per the RFC <a href="https://www.w3.org/Protocols/rfc2616/rfc2616.html" title="https://www.w3.org/Protocols/rfc2616/rfc2616.html">https://www.w3.org/Protocols/rfc2616/rfc2616.html</a>, http headers are case-insensitive, which means the destination application should be able to understand both the uppercase and lowercase.</p>
<blockquote>
<p>Each header field consists of a name followed by a colon (“:”) and the field value. Field names are case-<strong>in</strong>sensitive</p>
</blockquote>
<p>Since for most of the applications, http header is case-insensitive. Ruby’s standard networking library NET::HTTP converts every header to uppercase which is not an issue for most of the third party apis.</p>
<p>Many popular libraries like Httparty use Net::HTTP as backend.</p>
<p>But sometimes we need to send a particular http header as is, without any modifications.</p>
<p>To prevent http header keys from being modified by NET::HTTP, I used this solution from <a href="https://calvin.my/posts/force-http-header-name-lowercase" title="https://calvin.my/posts/force-http-header-name-lowercase">https://calvin.my/posts/force-http-header-name-lowercase</a></p>
<p>which worked fine.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">ImmutableKey</span> <span class="o"><</span> <span class="no">String</span>
<span class="k">def</span> <span class="nf">capitalize</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and using the above key as follows</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span> <span class="no">ImmutableKey</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s2">"api-key"</span><span class="p">):</span> <span class="s1">'SECRET-KEY'</span> <span class="p">}</span>
</code></pre></div></div>
<p>As per the third party, this started occurring from a particular time and then it occurred to me, the timeline matches perfectly with our rails upgrade from rails 4 to rails 5.
To verify the hunch, I ran the same code again in both rails 4 and rails 5 boxes with HTTParty debug log on and yep, there it was. Headers were being capitalized in new rails boxes.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Rails 4 box</span>
opening connection to thirdparty.com:443...
starting SSL <span class="k">for </span>thirdparty.com:443...
SSL established
<- POST <span class="s2">"/endpoint HTTP/1.1</span><span class="se">\r\n</span><span class="s2">api-key: 'SECRET'
</span></code></pre></div></div>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Rails 5 box</span>
opening connection to thirdparty.com:443...
starting SSL <span class="k">for </span>thirdparty.com:443...
SSL established
<- POST <span class="s2">"/endpoint HTTP/1.1</span><span class="se">\r\n</span><span class="s2">API-KEY: 'SECRET'
</span></code></pre></div></div>
<p>But why did it happen, my first thought was to check for gem version of httparty and even though there was a bump from <code class="language-plaintext highlighter-rouge">.0.14</code> to <code class="language-plaintext highlighter-rouge">0.17</code>, further debugging proved that httparty was not the issue and mutation of forms were happening at <code class="language-plaintext highlighter-rouge">Net::HTTP</code> level.</p>
<p><a href="https://github.com/jnunemaker/httparty/blob/99751ac98af929b315c74c2ac0f5ffa09195f7ae/lib/httparty/request.rb#L213" title="https://github.com/jnunemaker/httparty/blob/99751ac98af929b315c74c2ac0f5ffa09195f7ae/lib/httparty/request.rb#L213">https://github.com/jnunemaker/httparty/blob/99751ac98af929b315c74c2ac0f5ffa09195f7ae/lib/httparty/request.rb#L213</a></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">setup_raw_request</span>
<span class="vi">@raw_request</span> <span class="o">=</span> <span class="n">http_method</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">request_uri</span><span class="p">(</span><span class="n">uri</span><span class="p">))</span>
<span class="vi">@raw_request</span><span class="p">.</span><span class="nf">body_stream</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:body_stream</span><span class="p">]</span> <span class="k">if</span> <span class="n">options</span><span class="p">[</span><span class="ss">:body_stream</span><span class="p">]</span>
<span class="k">if</span> <span class="n">options</span><span class="p">[</span><span class="ss">:headers</span><span class="p">].</span><span class="nf">respond_to?</span><span class="p">(</span><span class="ss">:to_hash</span><span class="p">)</span>
<span class="n">headers_hash</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:headers</span><span class="p">].</span><span class="nf">to_hash</span>
<span class="vi">@raw_request</span><span class="p">.</span><span class="nf">initialize_http_header</span><span class="p">(</span><span class="n">headers_hash</span><span class="p">)</span>
<span class="c1"># If the caller specified a header of 'Accept-Encoding', assume they want to</span>
<span class="c1"># deal with encoding of content. Disable the internal logic in Net:HTTP</span>
<span class="c1"># that handles encoding, if the platform supports it.</span>
<span class="k">if</span> <span class="vi">@raw_request</span><span class="p">.</span><span class="nf">respond_to?</span><span class="p">(</span><span class="ss">:decode_content</span><span class="p">)</span> <span class="o">&&</span> <span class="p">(</span><span class="n">headers_hash</span><span class="p">.</span><span class="nf">key?</span><span class="p">(</span><span class="s1">'Accept-Encoding'</span><span class="p">)</span> <span class="o">||</span> <span class="n">headers_hash</span><span class="p">.</span><span class="nf">key?</span><span class="p">(</span><span class="s1">'accept-encoding'</span><span class="p">))</span>
<span class="c1"># Using the '[]=' sets decode_content to false</span>
<span class="vi">@raw_request</span><span class="p">[</span><span class="s1">'accept-encoding'</span><span class="p">]</span> <span class="o">=</span> <span class="vi">@raw_request</span><span class="p">[</span><span class="s1">'accept-encoding'</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>specifically this line</p>
<p><code class="language-plaintext highlighter-rouge">@raw_request.initialize_http_header(headers_hash</code></p>
<p>So if <code class="language-plaintext highlighter-rouge">NET::HTTP</code> was the issue here then why did rails upgrade break it ?</p>
<p>It was due to ruby version upgrade, earlier we were using ruby <code class="language-plaintext highlighter-rouge">2.1.3</code> and with the rails upgrade we jumped to ruby <code class="language-plaintext highlighter-rouge">2.5.2</code> which means standard library also had some changes.</p>
<p>So let see the diff between two versions</p>
<p>Older ruby version <code class="language-plaintext highlighter-rouge">2.1.3</code> had</p>
<p><a href="">https://github.com/ruby/ruby/blob/6d728bdae9de565ad9d0b2fee2d4c2a33c6f4eac/lib/net/http/header.rb#L162</a></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">each_capitalized</span>
<span class="nb">block_given?</span> <span class="n">or</span> <span class="k">return</span> <span class="n">enum_for</span><span class="p">(</span><span class="n">__method__</span><span class="p">)</span>
<span class="vi">@header</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span><span class="n">v</span><span class="o">|</span>
<span class="k">yield</span> <span class="n">capitalize</span><span class="p">(</span><span class="n">k</span><span class="p">),</span> <span class="n">v</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s1">', '</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">alias</span> <span class="n">canonical_each</span> <span class="n">each_capitalized</span>
<span class="k">def</span> <span class="nf">capitalize</span><span class="p">(</span><span class="nb">name</span><span class="p">)</span>
<span class="nb">name</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="sr">/-/</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span><span class="o">|</span><span class="n">s</span><span class="o">|</span> <span class="n">s</span><span class="p">.</span><span class="nf">capitalize</span> <span class="p">}.</span><span class="nf">join</span><span class="p">(</span><span class="s1">'-'</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>while <del>ruby</del> 2.5 <del>introduced</del> this commit <a href="https://github.com/ruby/ruby/commit/1a98f56ae14724611fc8f7c220e470d27f6b57e4" title="https://github.com/ruby/ruby/commit/1a98f56ae14724611fc8f7c220e470d27f6b57e4">https://github.com/ruby/ruby/commit/1a98f56ae14724611fc8f7c220e470d27f6b57e4</a> introduced some changes to underlying <code class="language-plaintext highlighter-rouge">captialize</code> method by using <code class="language-plaintext highlighter-rouge">to_s</code></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">name</span><span class="p">.</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="sr">/-/</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span><span class="o">|</span><span class="n">s</span><span class="o">|</span> <span class="n">s</span><span class="p">.</span><span class="nf">capitalize</span> <span class="p">}.</span><span class="nf">join</span><span class="p">(</span><span class="s1">'-'</span><span class="p">)</span>
</code></pre></div></div>
<p>which caused our ImmutableString class to return a new string object instead an object of ImmutableString class with capitalized frozen</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ImmutableKey</span><span class="p">(</span><span class="s2">"new"</span><span class="p">).</span><span class="nf">class</span> <span class="c1"># ImmutableKey</span>
<span class="no">ImmutableKey</span><span class="p">(</span><span class="s2">"new"</span><span class="p">).</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">class</span> <span class="c1"># String</span>
<span class="no">ImmutableKey</span><span class="p">(</span><span class="s2">"new"</span><span class="p">).</span><span class="nf">to_str</span><span class="p">.</span><span class="nf">class</span> <span class="c1"># String</span>
</code></pre></div></div>
<h1 id="fix">Fix</h1>
<p>Fix was to make the <code class="language-plaintext highlighter-rouge">to_s</code> and <code class="language-plaintext highlighter-rouge">to_str</code> return the <code class="language-plaintext highlighter-rouge">self</code> so that the returned object is an instance of <code class="language-plaintext highlighter-rouge">ImmutableKey</code> instead of the base string class</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">ImmutableKey</span> <span class="o"><</span> <span class="no">String</span>
<span class="k">def</span> <span class="nf">capitalize</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">to_s</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="kp">alias_method</span> <span class="ss">:to_str</span><span class="p">,</span> <span class="ss">:to_s</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Debugging the issue was fun though :D</p>
<p>If I missed out on something or something is not correct, then do let me know and I will correct it :)</p>Jatin DhankharAdventures with using custom http header and how ruby's NET::HTTP library parses itResponsible Disclosure - Codementor2018-07-09T00:00:00+00:002018-07-09T00:00:00+00:00https://jatindhankhar.in/blog/disclosure-codementor<h2 id="tldr">tl;dr</h2>
<p>Here is a tl;dr if you don’t have the time to go through whole post.
Found an open redirect exploit in Codementor’s auth flow, which doubled as a XSS exploit.
Also, an almost subdomain takover.</p>
<h2 id="the-bug">The Bug</h2>
<p>While logging into Codementor I noticed a <code class="language-plaintext highlighter-rouge">from</code> and <code class="language-plaintext highlighter-rouge">to</code> parameter in the auth-flow url. It piqued my interest,since redirect paramters are notorious for open redirect exploits.</p>
<h3 id="1-open-re-direct">1. Open Re-direct</h3>
<p>To test for open redirect bug, obvious place was to look at the <code class="language-plaintext highlighter-rouge">to</code> parameter since it controls where the user will be redirect. While testing my hopes were not high as it’s rare to find open redirects, but you never know until you try.
I crafted following url
<code class="language-plaintext highlighter-rouge">https://www.codementor.io/auth-redirect?from=https://www.codementor.io/login&to=https://www.google.com/</code> to redirect the user to <code class="language-plaintext highlighter-rouge">google.com</code> and, voila, it worked. After seeing many writeups on open redirect, I was happy to finally find and test one in the open (like a pokemon :smile:).</p>
<h3 id="2-xss">2. XSS</h3>
<p>Whenever there is open redirect, there is a tiny possiblity that it can be converted into XSS.
By subsituting the url in the <code class="language-plaintext highlighter-rouge">to</code> parameter with a javascript snippet we have the capability to execute arbitrarty code on the user’s machine.</p>
<p>Same auth flow can be exploited as XSS, if to parameter is replaced by javascript code</p>
<p><code class="language-plaintext highlighter-rouge">https://www.codementor.io/auth-redirect?from=https://www.codementor.io/login&to=javascript:alert(%271%27);</code></p>
<p>will execute <code class="language-plaintext highlighter-rouge">alert(1)</code> on the user’s machine. Of course it can be used to do cookie stealing and more mailcious tasks.</p>
<p><img src="/images/disclosure-codementor/xss.png" /></p>
<p>POC of the above issues</p>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/ejA0JGsfSsw?rel=0&ecver=1" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<h3 id="3-multiple-subdomain-takeover">3. Multiple Subdomain Takeover</h3>
<h4 id="this-isnt-clear-whether-it-was-real-issue-or-not-i-believe-it-was-codementor-thinks-it-wasnt-maybe-i-am-wrong-if-you-know-better-let-me-know">This isn’t clear whether it was real issue or not. I believe it was, Codementor thinks it wasn’t. Maybe I am wrong. If you know better, let me know.</h4>
<p>Looking at DNS entries it is evident that majority of Codementor infrastructure runs on Heroku.
While doing sub domain enumeration, I came across some domains that point to heroku instances that were available, so I grabbed them.</p>
<p>Some of the sub domains were <a href="https://community-api.codementor.io">community-api.codementor.io</a>, <a href="https://hire-api.codmentor.io">hire-api.codmentor.io</a>, <a href="https://event-api.codementor.io">event-api.codementor.io</a> and most of them pointed to an available domain on heroku <code class="language-plaintext highlighter-rouge">kochi-8109.herokussl.com</code>. Since the domain was available, I grabbed one for myself.
Domain I grabbed was <code class="language-plaintext highlighter-rouge">kochi-8109.herokuapp.com</code>.
However to confirm the theory I had to add cname entries to my domain which required having a paid plan which I tried to subsribe but Heroku couldn’t recognize/process my Debit card :disappointed: .</p>
<p><img src="/images/disclosure-codementor/heroku.png" />
So I reported the issue as it is.</p>
<p>Before reporting the issue Codementor’s DNS entries were as following. Most of them pointed to <code class="language-plaintext highlighter-rouge">****.herokussl.com</code></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">host community-api.codementor.io ✘ 130
community-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>kochi-8109.herokussl.com.
kochi-8109.herokussl.com is an <span class="nb">alias </span><span class="k">for </span>elb049633-259037725.us-east-1.elb.amazonaws.com.
elb049633-259037725.us-east-1.elb.amazonaws.com has address 174.129.236.6
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.229.39
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.232.248
host event-api.codementor.io
event-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>kochi-8109.herokussl.com.
kochi-8109.herokussl.com is an <span class="nb">alias </span><span class="k">for </span>elb049633-259037725.us-east-1.elb.amazonaws.com.
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.229.39
elb049633-259037725.us-east-1.elb.amazonaws.com has address 174.129.236.6
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.232.248
host hire-api.codementor.io
hire-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>kochi-8109.herokussl.com.
kochi-8109.herokussl.com is an <span class="nb">alias </span><span class="k">for </span>elb049633-259037725.us-east-1.elb.amazonaws.com.
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.232.248
elb049633-259037725.us-east-1.elb.amazonaws.com has address 23.23.229.39
elb049633-259037725.us-east-1.elb.amazonaws.com has address 174.129.236.6</code></pre></figure>
<p>After reporting the issue they abruptly changed the DNS entries. Most of them now point to <code class="language-plaintext highlighter-rouge">****.herokudns.com</code></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"> host community-api.codementor.io
community-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>community-api.codementor.io.herokudns.com.
community-api.codementor.io.herokudns.com has address 52.203.53.176
community-api.codementor.io.herokudns.com has address 52.21.108.248
community-api.codementor.io.herokudns.com has address 52.207.5.158
community-api.codementor.io.herokudns.com has address 52.207.39.76
community-api.codementor.io.herokudns.com has address 52.22.127.224
community-api.codementor.io.herokudns.com has address 52.55.191.55
community-api.codementor.io.herokudns.com has address 52.22.2.149
community-api.codementor.io.herokudns.com has address 52.7.126.198
host hire-api.codementor.io
hire-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>hire-api.codementor.io.herokudns.com.
hire-api.codementor.io.herokudns.com has address 52.207.5.158
hire-api.codementor.io.herokudns.com has address 52.22.2.149
hire-api.codementor.io.herokudns.com has address 52.22.127.224
hire-api.codementor.io.herokudns.com has address 52.23.126.223
hire-api.codementor.io.herokudns.com has address 52.22.213.157
hire-api.codementor.io.herokudns.com has address 52.21.108.248
hire-api.codementor.io.herokudns.com has address 52.3.63.2
hire-api.codementor.io.herokudns.com has address 52.4.95.48
host event-api.codementor.io
event-api.codementor.io is an <span class="nb">alias </span><span class="k">for </span>event-api.codementor.io.herokudns.com.
event-api.codementor.io.herokudns.com has address 52.44.53.64
event-api.codementor.io.herokudns.com has address 52.207.5.158
event-api.codementor.io.herokudns.com has address 52.44.230.61
event-api.codementor.io.herokudns.com has address 52.55.191.55
event-api.codementor.io.herokudns.com has address 52.203.53.176
event-api.codementor.io.herokudns.com has address 52.5.182.176
event-api.codementor.io.herokudns.com has address 52.207.39.76
event-api.codementor.io.herokudns.com has address 52.7.126.198</code></pre></figure>
<p>After reporting the issue, this was Codementor’s reply</p>
<blockquote>
Hi Jatin,
Sorry for the late reply. One of the mentors on our platform reminded us of these vulnerability. We were able to fix the issues with Open Re-direct and XSS. However, our website are not pointed to domains like xxx.herokuapp.com, so there won't be a problem if that domain is experiencing a takeover.
Since when you sent in the report, these issues weren't fixed yet or reported publicly by us. We are offering a total of $50, do you have a PayPal account that we can transfer to?
</blockquote>
<p>Codementor acknowledged the first two issues but didn’t acknowledge the last issue (sub domain takeover) as a valid one, even though they changed the DNS entries (why change the entries if it’s okay :confused: ?)</p>
<p>I asked them to re-consider the third report and provided the proof ( sudden change in the DNS entries).
Their reply was</p>
<blockquote>
Hi Jatin,
Sorry for the confusion, but the change was made for operation purposes, it was a switch for the services they provided.
The third issue you have reported actually was pointing to the domain herokussl.com, not the herokuapp.com you reserved. Since we are only considering the first two issues, that's why we are offering a total of $50. Thanks for understanding
</blockquote>
<p>AFAIK, had I been able to subscribe to paid plan and add cname, I could have redirected traffic from <code class="language-plaintext highlighter-rouge">**.herokussl.com</code> or maybe Codementor team was right. If you know what went wrong, let me know in the comments below (damn, this sounds like a fellow Youtuber :laugh:)</p>
<h2 id="bounty">Bounty</h2>
<p>I received a bounty of $50. This may not sound like a lot and I negotiated with Codementor.</p>
<p>But this was my first paid reward apart from swag, so yeah, not much to complain about.</p>
<h2 id="thanks">Thanks</h2>
<p>Thanks to Codementor for fixing the bugs and permission to disclose the issues.</p>Jatin DhankharResponsible Disclosure - HackerRank2018-05-16T00:00:00+00:002018-05-16T00:00:00+00:00https://jatindhankhar.in/blog/disclosure-hackerrank<h2 id="tldr">tl;dr</h2>
<p>Here is a tl;dr if you don’t have the time to go through whole post.</p>
<p>Found a way to read other user’s submissions via Code Editor.</p>
<h2 id="how-i-found-the-issue">How I found the issue</h2>
<p><a href="https://www.hackerrank.com/">HackerRank</a> is a online programming paltform to learn,compete and find jobs. Like many online code platform, first thing to look for is the code editor and check if it can be exploited.</p>
<p>I was able to do directory traversal and read some files and it was successful. I was able to read <code class="language-plaintext highlighter-rouge">passwd</code> files, read Android KeyStore files. Excited with my new findings, I contacted the HackerRank support and it turned out be a feature :neutral_face:</p>
<p>This was their response</p>
<blockquote>
The files /etc/resolv.conf, /etc/passwd and few jar files and a C++ header file are provided with read-only access to the user. You are allowed to do file traversal and read files. Our execution environment allows those operations, with certain limitations.
We allow this access to all system files so that all code submissions can run correctly and can use standard library functions.
Certain files can be modified and are allowed because of certain use-cases. For example, if someone wants to store the code state in a file at one point and can read it back later at a different point of execution.
If you find a way to manipulate the system files or can read other user's submissions, then that's a valid vulnerability, and if you have discovered such an exploit, then it will be considered for the bounty program.
</blockquote>
<p>Turns out, it was a feature and It was a false positive. I come around many false positives :sweat_smile:</p>
<p>However, If I can find a way to read other user submissions then it’s a valid vulnerability and eligible for bounty, so in short,if I can read other’s code then I am good to go :smile:</p>
<p>I was about to give up because I was tired, but I wanted to give it one more shot before throwing in the towel.</p>
<p>I tried to read logs, but there were none that could be accessed via code editor. If somehow I can read logs or list of files that are open for each users I can do something.
When a user submits a code, a random user on the machine is assinged, usually it’s of the form <code class="language-plaintext highlighter-rouge">execution-user-xxxxx</code>
but if we check the working directory of users it’s of the following form <code class="language-plaintext highlighter-rouge">/run-xxxxxxxxxxxx</code></p>
<p><img src="/images/disclosure-hackerrank/user_files_structure.png" /></p>
<p>But I checked this after discovering another way to read the submissions.
I originally wanted to know what files were open during the code execution which can point me in the right direction.
So I tried the <code class="language-plaintext highlighter-rouge">lsof</code> command which returns <strong>list</strong> of <strong>open files</strong></p>
<p><img src="/images/disclosure-hackerrank/lsof_output.png" /></p>
<p>and all the user submissions were on the root level</p>
<p><img src="/images/disclosure-hackerrank/root_structure.png" /></p>
<p>Structure of a user directory is of following form</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/run-<span class="k">******************</span>/:
compile.err
error00006.err
error00007.err
input
input00006.in
input00007.in
output00006.out
output00007.out
request.json
response_part.json
solution
solution.cc
</code></pre></div></div>
<p><strong>If we can read solution.*, we can find solutions that were submitted roughly around the same time period, plus reading input000xxx.in we can find out potential test cases to any problem and cheat our way through</strong> :smiling_imp:</p>
<p>Let’s read the the solutions submitted for <code class="language-plaintext highlighter-rouge">c</code> .</p>
<p><code class="language-plaintext highlighter-rouge">less /run-*/solution.c</code></p>
<p><img src="/images/disclosure-hackerrank/c_solution.png" /></p>
<p>what if we don’t the know extension of the solution we can either do <code class="language-plaintext highlighter-rouge">less /run-*/solution.*</code> or just print the <code class="language-plaintext highlighter-rouge">request.json</code> as it contains solution submitted as well some other meta data.</p>
<p><code class="language-plaintext highlighter-rouge">less /run-*/request.json</code></p>
<p><img src="/images/disclosure-hackerrank/json_response.png" /></p>
<p>Sadly we cannot see <code class="language-plaintext highlighter-rouge">response.json</code> due to permission issues, otherwise we could have extracted more juicy info (?)</p>
<p>If we want to know hidden test cases for a problem, no problemo, just do <code class="language-plaintext highlighter-rouge">less input*.in</code></p>
<p>When I had enough evidence that I could read user submissions, I submitted the same. They confirmed the same and asked me to explore further possibilites, if I can do manipulate files or link back each solution to the user.</p>
<blockquote>
We see that that the process id and the execution-user XXXX belongs to the sandbox environment the code is running in. Please explore further and see if you can get more other user info or exploit our code checker or hack into the server, we will gladly pass on the information to our team.
</blockquote>
<p>I tried out some things, looked for environment variables,deep directory traversal, causing kernel panic.
There were some dangerous things like dumping a zip bomb which I didn’t try, but hinted about all the possibility attacks in theory but there was nothing more I could think to attack. Hence I threw the towel.</p>
<p>This was HackerRanks’ final response</p>
<blockquote>
As of now, we do not see a major vulnerability, but for all your effort we would like to show our appreciation by sending hackerrank T-Shirt your way
</blockquote>
<h2 id="bounty">Bounty</h2>
<p>Since I wasn’t able to manipulate system files or link back a solution to a user, no bounty was rewarded. But HackerRank appreicated my effort and decided to send a cool T-shirt my way
So no reward but a cool T-shirt :smile:</p>
<p>Gifts are always appreciated :smile:</p>
<h2 id="timeline">Timeline</h2>
<ul>
<li>9 May, 2018 [11:01 PM IST] - Contacted the Customer Support and reported the issue.</li>
<li>10 May, 2018 [2:54 AM IST] - Response from HackerRank rep.</li>
<li>10 May, 2018 [4:07 AM IST] - Response from HackerRank’s Techincal Solutions Engineer. Initial report was false positive. Was told to look for a way to read user submissions</li>
<li>10 May, 2018 [9:33 PM IST] - Found a way to read user submissions. Reported the same</li>
<li>11 May, 2018 [11:14 AM IST] - Confirmation from HackerRank. Asked if I can explore further</li>
<li>11 May, 2018 [11:31 PM IST] - Coudln’t find a way to manipulate files and tie back submission to a user.</li>
<li>10 May, 2018 - Asked to disclose the report after the fix.</li>
<li>11 May, 2018 [3:21 PM IST] - Double Confirmed the fix.</li>
<li>14 May, 2018 [11:31 AM IST] - Asked for an update.</li>
<li>14 May, 2018 [10:35 PM IST] - Report wasn’t eligible for bounty, but got a T-shirt</li>
<li>15 May, 2018 [12:06 PM IST] - Asked to disclose the issue</li>
<li>16 May, 2018 [12:02 AM IST] - Recieved confirmation</li>
<li>16 May, 2018 [11:35 PM IST] - Shared the draft</li>
<li>18 July, 2018 [2:56 AM IST] - Draft approved</li>
<li>18 July, 2018 [2:16 PM IST] - Draft published</li>
</ul>
<h2 id="thanks">Thanks</h2>
<p>I contacted HackerRank support and they were prompt,helpful and supportive during the whole process :+1:.
So far all the companies I have contacted were quite nice, even when my reports were invalid or false positives.</p>Jatin DhankharReading other user`s code submissions - HackerEarthResponsible Disclosure - Hacker Earth2018-05-15T00:00:00+00:002018-05-15T00:00:00+00:00https://jatindhankhar.in/blog/disclosure-hackerearth<h2 id="tldr">tl;dr</h2>
<p>Here is a tl;dr if you don’t have the time to go through whole post.
Found an open un-configured worpdress sub blog and taking over it over to do Remote Code Execution and more.
Issue has been fixed by HackerEarth.</p>
<h2 id="how-i-found-the-issue">How I found the issue</h2>
<p>While doing subdomain enumeration for <a href="https://hackerearth.com">https://hackerearth.com</a>, I came across <a href="http://demo-wordpress.hackerearth.com">http://demo-wordpress.hackerearth.com</a>
which had directory listing of many sub blogs.
<img src="/images/disclosure-hackerearth/blog_listings.png" /></p>
<p>Seeing directory listing pointed to possibily that something is mis configured, so I started opening every sub blog link and behold, there it was, an unconfigured wordpress instance, ready to be taken over, <code class="language-plaintext highlighter-rouge">innovation_from_recurit</code> blog was open and ready to be configured.
Here is how it looked like</p>
<p><img src="/images/disclosure-hackerearth/wp_installation.png" /></p>
<p>I followed the installation and was able to gain admin rights to the following sub blog.</p>
<p><img src="/images/disclosure-hackerearth/wp_installed.png" /></p>
<p>Now, this in itself is an issue worth reporting but exploring it within bounds never hurts and helps make the report more worthy.
One further exploration, it turns out, wordpress is used as landing pages for new hackathons. So Hackerearth uses Wordpress extensively and there is always a possiblity that some one re-uses the same password, across many systems.
Now I needed to explore the system without shell access, thankfully, wordpress comes with lots of useful plugins that can be installed, usually they provide a convenient way to perform common tasks.</p>
<p>To access Database, I used <strong>Adminer</strong> (which sounds like a malware to mine crypto currency :sweat_smile:) and I was able to explore database for other sub blogs as well, since they all were running on the same system.</p>
<p><img src="/images/disclosure-hackerearth/adminer_panel.png" /></p>
<p>Accessing proper table, all the information, including user_names,email address and hashed password were accessible.
Although I couldn’t recover the original passwords, but it was easy to reset the password of any user and post content on behalf of that user.</p>
<p><img src="/images/disclosure-hackerearth/adminer_blog_users.png" /></p>
<p>Using <strong>File Manager Advanced</strong> to explore file system</p>
<p><img src="/images/disclosure-hackerearth/file_explorer.png" /></p>
<p>Using <strong>Developer Tools</strong> to get system info</p>
<p><img src="/images/disclosure-hackerearth/developer_tools.png" /></p>
<p>and finally using <strong>WpTerm</strong> to execute some commands (commands that require a proper tty interface don’t work), which can be further used to inject backdoors.</p>
<p><img src="/images/disclosure-hackerearth/rce.png" /></p>
<h2 id="bounty">Bounty</h2>
<p>Although HackerEarth have a bug bounty program <a href="https://www.hackerearth.com/docs/wiki/program/bounty/">https://www.hackerearth.com/docs/wiki/program/bounty/</a>, they are not providing rewards yet :neutral_face:</p>
<p>Instead, I will be a getting a cool T-shirt :smile: .</p>
<p>Aiming to have a new tech Tee for each day of a month :sweat_smile: .</p>
<h2 id="timeline">Timeline</h2>
<ul>
<li>2 May, 2018 [3:18 PM IST] - Contacted the Customer Support where to report the issue</li>
<li>2 May, 2018 [3:21 PM IST] - Response from customer support to sent the details.</li>
<li>2 May, 2018 [3:33 PM IST] - Sent the details</li>
<li>4 May, 2018 - Asked for an Update. Said they are looking into it</li>
<li>8 May, 2018 - Response from concerened Team, regarding the fix. Fix was partial only listing was removed, blog was still there. Told them so</li>
<li>9 May, 2018 - They removed the blog but forgot to reply back (?)</li>
<li>10 May, 2018 - Asked to disclose the report after the fix.</li>
<li>11 May, 2018 [3:21 PM IST] - Double Confirmed the fix.</li>
<li>11 May, 2018 [7:12 PM IST] - HackerEarth Rewarded T-Shirt as bounty :)</li>
<li>15 May, 2018 [20:30 PM IST] - Shared the Disclosure draft with HackerEarth</li>
<li>18 May, 2018 - Draft approved and post Published.</li>
</ul>
<h2 id="thanks">Thanks</h2>
<p>I contacted HackerEarth support and they were quite fast,helpful and supportive during the whole process :+1:.</p>Jatin DhankharTaking over HackerEarth Sub Blog to RCE- HackerEarthResponsible Disclosure - 1Mg2018-04-30T00:00:00+00:002018-04-30T00:00:00+00:00https://jatindhankhar.in/blog/disclosure-1mg<p>I recently started discovering and learning about the world of Network Security and Bug Bounty, and it’s absolutely addicting, like playing a game. Wandering around to find treasures, getting defeated, accomplishing levels and sometimes getting rewarded.</p>
<h2 id="tldr">tl;dr</h2>
<p>Here is a tl;dr if you don’t have the time to go through whole post.
While doing subdomain enumeration I found <a href="https://dmg.1mg.com/html/login.html">dmg.1mg.com</a> and found a Google Search API key and an unauthenticated API endpoint.</p>
<h2 id="how-i-found-the-issue">How I found the issue</h2>
<p>While doing subdomain enumeration for <a href="https://www.1mg.com/">https://www.1mg.com/</a>, I came across their in-house Drug Management System <a href="https://dmg.1mg.com">dmg.1mg.com</a>. By default, page redirects to <code class="language-plaintext highlighter-rouge">login.html</code>, let’s try signing up <a href="https://dmg.1mg.com/html/signup.html">https://dmg.1mg.com/html/signup.html</a>, nope no luck, there is no signup page.</p>
<p>Then I looked at Network requests being made, and noticed a GET reuest for <code class="language-plaintext highlighter-rouge">config.json</code>
<img src="/images/disclosure-1mg/network-tab.png" />
and it contained API keys and endpoints.
Here’s what <code class="language-plaintext highlighter-rouge">config.json</code> contained</p>
<p><img src="/images/disclosure-1mg/config-json.png" /></p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w">
</span><span class="p">{</span><span class="w"> </span><span class="nl">"SEARCH_ENGINE_KEY"</span><span class="p">:</span><span class="w"> </span><span class="s2">"014227405161808294265:4qaxa_56cxk"</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="nl">"TICKET_LIMIT"</span><span class="p">:</span><span class="w"> </span><span class="s2">"20"</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="nl">"DMG_API"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://api.1mg.com/dmg"</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="nl">"GOOGLE_KEY"</span><span class="p">:</span><span class="w"> </span><span class="s2">"AIzaSyAREDxwRpknZvq-mKQ_vadaMnNDP3_rdbk"</span><span class="p">,</span><span class="w"> </span><span class="nl">"LABS_TEST_API"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://api.1mglabs.com/admin/test"</span><span class="w"> </span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>and I was like.
<a href="https://imgflip.com/i/1pg3b2"><img src="https://i.imgflip.com/1pg3b2.jpg" title="made at imgflip.com" /></a></p>
<p>There was one problem though, I don’t know where these keys are used. Without having a possible attack secnario there is no point of reporting it.</p>
<p>Looking at <code class="language-plaintext highlighter-rouge">GOOGLE_KEY</code> and <code class="language-plaintext highlighter-rouge">SEARCH_ENGINE_KEY</code>, it was hinting at a Google Service that uses search engine key, and then I found it, it was <a href="https://developers.google.com/custom-search/json-api/v1/overview">Google Custom Search API</a>.</p>
<p>Looking at documentation, I was able to figure out how to consume the api.
Let’s make a GET request to <code class="language-plaintext highlighter-rouge">https://www.googleapis.com/customsearch/v1?q={QUERY}&key={API_KEY}</code> but complains about missing paramter cx (The custom search engine ID to scope the search query (string)), so cx is the <code class="language-plaintext highlighter-rouge">SEARCH_ENGINE_KEY</code>.</p>
<p>Let’s craft the new GET request as follows <code class="language-plaintext highlighter-rouge">https://www.googleapis.com/customsearch/v1?q={QUERY}&key={API_KEY}&cx={CUSTOM_KEY}</code></p>
<p>So I made a GET to the following endpoint
<code class="language-plaintext highlighter-rouge">https://www.googleapis.com/customsearch/v1?q=jatin dhankhar&key=AIzaSyAREDxwRpknZvq-mKQ_vadaMnNDP3_rdbk&cx=014227405161808294265:4qaxa_56cx/k</code></p>
<p>and I got following in the response.</p>
<p><img src="/images/disclosure-1mg/final-query.png" /></p>
<p>I tried looking for 1mg’s bug bounty/security page but there was none, contacted them via <code class="language-plaintext highlighter-rouge">security@1mg.com</code>, turns out email didn’t exists.</p>
<p>So I sent them a DM on Twitter and they said to send the report to <code class="language-plaintext highlighter-rouge">care@1mg.com</code> and they will forward to tech team. 1Mg’s customer care was very prompt and supportive :+1:.</p>
<p>Since Google Custom Search has a paid varaint , this was a financial risk.
5$ for 1000 queries, [which can be done under a minute or two, on a multi threaded system]</p>
<p>1mg fixed the issue by removing and revoking the <code class="language-plaintext highlighter-rouge">GOOGLE_KEY</code> and <code class="language-plaintext highlighter-rouge">SEARCH_ENGINE_KEY</code>.</p>
<p>Next I turned my attention to the remaining part of the <code class="language-plaintext highlighter-rouge">config.json</code>. I opened the <code class="language-plaintext highlighter-rouge">LABS_TEST_API</code> url <code class="language-plaintext highlighter-rouge">https://api.1mglabs.com/admin/test</code> and it said <code class="language-plaintext highlighter-rouge">{"error": "Required params search_text not found"}</code>. I thought, “Hmm, Interesting, let’s give the endpoint what it wants, a search_text parameter”.
So I added the <code class="language-plaintext highlighter-rouge">search_text</code> parameter with <code class="language-plaintext highlighter-rouge">para</code> as query (to look for paracetamol :pill:).
Final url was <code class="language-plaintext highlighter-rouge">https://api.1mglabs.com/admin/test?search_text=para</code> and response was this.</p>
<p><img src="/images/disclosure-1mg/dmg-response.png" /></p>
<p>Again my reaction was this</p>
<p><a href="https://imgflip.com/i/1pg3b2"><img src="https://i.imgflip.com/1pg3b2.jpg" title="made at imgflip.com" /></a>.</p>
<p>I contacted them again and reported the issue and they fixed the issue by adding authentication on the top of the api.</p>
<h2 id="bounty">Bounty</h2>
<p>Got a thanks from 1Mg :smile:.
No rewards, since they don’t have bug bounty program, so no <strong>₹</strong>.</p>
<h2 id="timeline">Timeline</h2>
<ul>
<li>27 April, 2018 - Reported API key issue to 1 Mg Customer Care</li>
<li>28 April, 2018 - 1 Mg Fixed the issue</li>
<li>28 April, 2018 - Reported open API endpoint issue to 1 Mg</li>
<li>30 April, 2018 - 1 Mg Fixed the issue</li>
<li>30 April, 2018 - Shared the disclosure draft for approval</li>
<li>8 May, 2018 - Draft approved and blog published.</li>
</ul>
<h2 id="thanks">Thanks</h2>
<p>Thanks to 1Mg for fixing the bugs and permission to disclose the issues.
1Mg customer care was prompt and supportive, :+1:</p>Jatin DhankharFinding API Keys in the open plus Unauthenticated API consumption - 1Mg