Notes on "Code BEAM V 2020" Authenticated uplinks with verdaccio Testing in Node.js by example using the SOLID principles Clean up Mac OS: How I freed 35GB of space Fixing 431 Request Header Fields Too Large in Node.js visit the /archive to see all 137 posts Setting up a Verdaccio npm registry Privacy and Coherence Elixir trick: start an Observer window with mix Validate your RSS feed Minimal dark mode with CSS and JavaScript Using Β΅compress to dramatically optimize static assets Ad blocking with Raspberry Pi and Pi-hole Optional chaining in Node.js 14 Nullish coalescing in Node.js 14 Road to Elixir: Monitor Crypto assets visit the /archive to see all 137 posts Resuming Elixir by self-hosting plausible analytics devblog: yet another static site generator, seriously Boring software development Full list of Chromium Puppeteer flags and command line switches How to connect puppeteer to a Proxy

Visit the archive to see all 137 posts

Subscribe to my newsletter

Bi-weekly email to stay up to date with #elixir #nodejs #agile #testing #refactoring #cleancode

Below you can skim through featured articles I wrote over the years.

To see all 137 articles, head over to the archive

Elixir, node.js, crypto, testing, tutorials, thoughts and more.

Notes on "Code BEAM V 2020"

Found a typo? Edit this page on GitHub

Written on   2020-05-28

2218 words - 17 minutes πŸ•œ

What: Code BEAM V official website

When: 2020/05/28 - 2020/05/29

Where: On the interwebz

Schedule: code-beam-v#schedule

Table of talks:

Day 1

Day 2

Opening Keynote - The Future of Programming

Talk 1 ~ 15:00 CEST

assets/images/posts/beam-v/222-participants.png

"It was crazy building soft near-realtime system 20 years ago" ~ Casarini

"Erlang is actually a Domain-Specific-Language written in C, tell your managers that and they'll buy that"

JVM for speed and parallelism.

BEAM for scalability and concurrency.

The predecessor of the BEAM was the JAM: Joe's Abstract Machine!

Looking at the future

Distribution: no worries about threads breaking, corrupting your memory, introduces latency, but is pretty fast at the end of the day.

The future is distributed.

BEAM

The BEAM can be seen as an OS with a hypervisor on top of your OS.

assets/images/posts/beam-v/beam-os.png

The BEAM has always been "cloud-native", not relying on particular OS or distribution.

The BEAM helps to abstract away from Distribution, abstract away from OS.

Letting you Focus on business logic.

assets/images/posts/beam-v/osi-1.png

You should be able to focus mostly just on the business side of programming, the actual business logic.

Erlang/Elixir helps you with that by e.g. using a gen-server and implementing just call or cast calls, without spending time on the underlying complexity.

Programming Model

Many different programming models

  • Functional Reactive
  • Lamdbas
  • Event-driven
  • Pub/Sub
  • Actor Model

OTP will let you scale both vertically and horizontally, with a simple programming model.

assets/images/posts/beam-v/osi-2.png

Abstract the infrastructure

Hopefully in the future, we won't be talking about Kubernetes, Lambdas, Docker-compose and abstract away the infrastructure.

Let's stop talking about tools, let's focus on valid, rock-solid abstractions.

This will help to avoid making software developers also Network Engineers.

If you don't have the correct abstractions in place, you are going to have problems evolving with time.

If you don't have a tool that is abstracted away on layer 7, let it be.

assets/images/posts/beam-v/osi-other.png

"The future is concurrent, distributed and properly abstracted!"

Images kindly provided by Francesco Cesarini


Building adaptive systems

Talk 2 ~ 15:55 CEST - Chris Keathley

Request spikes happen. Overloads can be nightmares. Latencies are introduced.

This can happen for both internal and external services, DB, APIs, etc.

"All services have objectives". For example requests/second

A resilient service should be able to withstand a 10x traffic spike and continue to meet those objectives.

Queues and Overloads

Most systems boil down to queues.

It contains work that needs to be processed.

Overload happens:

Arrival Rate (how quick items show up) > Processing Time (how much it takes to process an item)

Little's Law

Elements in the queue = Avg Arrival Rate * Avg Processing Time

CPU pressure happens when too many BEAM processes are created, and not processed in time to empty the queue.

This slow down the queue and scheduler of the BEAM and things can fall apart.

Overload Mitigation

You need to get Arrival Rate and Processing Time under control.

It would be obvious to start dropping items from the queue.

The server that processes the queue items could drop requests, or queue items themself and eventually evict items. (Downstream solution)

You could also stop sending requests all together to the downstream server (Upstream solution). Mitigated the load on the downstream.

Autoscaling

Autoscaling is not a silver bullet.

If you DB is under load, and you auto-scale your server, you just made things worse.

You need to keep in mind the Load shedding into the equation.

Circuit Breakers

If a server is under load, you can shut off the traffic to that server and let it "heal".

Circuit breakers are your last line of defense.

Circuit breakers should be dynamic and adapted your domain. You cannot have a static circuit breaker in a dynamic domain of yours.

Adaptive Concurrency

Read papers about Congestion Avoidance and Control.

Resilient to failures, a self-healing internet and systems.

Dynamically discover Adaptive limits by probing your system and seeing the actual limit of services.

Additive Increase Mutliplicative Decrease

You backoff much faster than you grow.

Tools and ideas: fuse and regulator (on github).

Backpressure

Backpressure can work great for internal services, but for e.g. spike in users your system needs to dynamically adapt to the circumstances.

Adopting Erlang, Adapting Rebar

It's easy to pick up a book, read the theories, but often get stuck in the more practical stuff.

Adopting Production Erlang:

with docker

  • efficient building
    • cache deps with rebar.lock
    • store local hex package cache
  • runtime concerns
    • busy wait (Erlangs' scheduler goes to sleep with a tight loop, burning your CPU)
    • schedulers
    • zombie processes

Most of the issues are fixed in OTP-23 and rebar 3 3.14

with kubernetes

Similar concerns as with docker.

You will get throttled if you reach certain CPU limits.

relx

Predecessor of rebar3.

Previously standalone escript.

Slimmed down and sped up, simplified configurations.

Dev, prod, and minimal mode.

Elixir meets Erlang in Blockchain development

Checkout Aeternity Blockchain!

Trust and Useability

Blockchains are trustless, distributed state-network.

Less cool: speed is terrible, useability is often quite bad.

This because useability comes to the cost of given trust to a certain authority.

Architecture

The FSM (finite state machine) handles channel protocol.

Watcher detects on-chain activity.

State Cache helps restore off-chain state.

assets/images/posts/beam-v/elixir-blockchain.png

State Channels

Created Off-chain. Speed, scalability. Still Trustless.

Off-chain as in "no-chain".

You pass tokens back and forth until the contract is concluded. It could take 10 minutes or 6 months.

Your smart contract is the "protocol" you're defining. Deploy the contract in the state channel.

coin-toss casino example

assets/images/posts/beam-v/coin-toss.png

More info here on github.com/aeternity and aeternity.com

Elixir update

Current version Elixir 1.10, January 2020. 900 contributors. 10k+ packages on hex.pm. 1.3B+ downloads.

Erlang/OPT21+ requirement

This because Elixir fully integrates with Erlang's new logger, everything is shared.

New guards: is_map_key/2 and is_struct/1.

Compilation tracers

Compilation in Elixir is the same as execution code.

This is because you can conditionally define functions and modules.

defmodule MyMod do
  if OtherMod.some_condition? do
    def some_fun do
      ...
    end
  end
end

This is where compilation tracers come into play. Receive a bunch of events (compile module, define functions, etc.).

Useful for static code analysers.

Important foundation for the language.

Compilation environment

Application environment is read at compile time, not at run time.

assets/images/posts/beam-v/compile-env.png

You can now use Application.compile_env to read variables at run time.

ExUnit Pattern Diffing

If you're interested in just a few fields of a struct, now Eixir gives you more readable traces.

assets/images/posts/beam-v/ex-unit-diffing.png

Future

1.11 in October 2020!

Calendar.strftime/3 for datetime formatting

New log levels, notice, critical alert and emergency.

Warnings if using modules from non-dependencies.

Read Erlang docs from the Elixir Shell!

Phoenix LiveDashboard

Comes with every new phoenix application (v1.5 and up).

Request logging, metrics etc.

Closing Keynote - The Tyranny of Structurlessness

How more meaningful code can make your project more resilient & maintainable

People want more and more from applications as times goes on!

-> Complexity grows at an exponentional rate.

-> more flexible and easier to scale

Let's focus on domain and structure!

Good elixir

Functional core, imperative shell

Inner data manipulation in clean isolated environment, actions runs on the outer layer (side-effects).

Another leayer between the outer layer and functional core: Semantic DSL / OO

Decouple imperative outer shell with inner pure-logic function core.

Brings to higher reuse of code.

Testing

Prop + model testing for function core.

Huge gains and cleaner code.

Tradeoffs

  • Exchange granular control for structure
  • humans over machines
  • meaning over mechanics
  • safer!

Actor Abyss

Each step is very simple in an actor-based application.

Reasoning about dynamic organisms is difficult.

Complexity grows faster.

Composition

Composition is at the heart of modularity

Orthogonality is at the heart of composition

assets/images/posts/beam-v/composition.png

no reinventing the wheel

GenServers etc are pretty low level! Add semantics to them!

A common example

def get(map, key, default \\ nil)

%{a: 1} |> Map.get(:b, 4)
#=> 4

def fallback(nil, default), do: default
def fallback(val, _), do: value

%{a: 1} |> Map.get(:b) |> fallback(4)
#=> 4

[] |> List.first() |> fallback(:empty)
#=> :empty

So instead of adding a third parameter to every function that implements the Enumerable protocol, you can abstract that semantic away!

good interfaces != good abstractions

Find a common interface with higher higher semantic density (focused on meaning not mechanics)

Define front-end and back-end interfaces well (could be sync and async!)

Declarative, configurable data flow, super extensible:

defimple Dataflow, for: %Stream{}
defimple Dataflow, for: %Distributed{}
defimple Dataflow, for: %Broadway{}

Summary

Protocols super useful for DDD

Add a semantic layer to your application code, based on your domain

Test your distributed system by looking at the properties

Prop-testing useful for structured abstractions

Opening Keynote - Problem led Software Design

Boyd Multerer and Robert Virding

The Erlang Problem

How can we improve the programming of Telephone applications? Very complex to build and maintain.

  • Handling very large number of concurrent activities
  • perform certain actions within a certain time
  • system distributed over several computers
  • maintainance without stopping the system
  • fault tolerance for hardware failures and software errors

These are not problems just for telecom!

Internal development

  • many threads at the same time
  • understanding and evolving the problem domain
  • designing language and arch
  • testing the idea and how it would perform

Erlang tested in Ericsson in project ACS/Dunder. The first users of Erlang in a real product

Doing lots of experiments and testing in the real world.

First principles

Lightweight concurrency, async communication. Process isolation (no effect on other processes), error handling (detect and handle errors), with soft real-time, non-blocking features.

Language/system should be simple, with a high level language to get real benefits.

Provide tools for building systems, not solutions. Basic operations needed for building communication protocols and error handling.


"Build for the future", so that your solutions don't bring to new problems, but the other way around.

Try to think of the problems you're not solving.

Think about the Actual problems, problems of purpose.

Problem Discovery! In Erlang they ahd the problems of the platform, because to test the platform in real live you need a real life problem and application. Kinda chicken-and-egg problem.

Scenic

By XBOX founder

OTP based control hierarchy, fast process based restarts

Jelly theory of development

Software is never really finished. If you touch it, it's gonna mold, change and adapt.

It's a continuous development and hacking target, Discover new problems emerging from usage.

Checkout key10.com and Scenic!

How the BEAM will change your mind

By Laura M. Castro

Like when in functional programming world you say, "you need to change your mind".

On an abstract level FP advocates sound fine, but getting more practical and approaching it is a different thing.

The imperative approach

"The God complex"

You see all data, you have control over all of it, you know it all (algorithm).

The object-oriented approach

"The Despicable master"

Small buckets of data, the bucket know about themselves.

The buckets don't know about each other. They can be seen as minions that do small tasks on their own data.

A master is still needed to orchestrate all together.

The functional approach

"The BEAM approach"

Each process has their own duties, can go bad but the world needs to go on. The processes can be supervised and monitored.

Eventually all characters work together and a result, a stable situation is reached.

Example: Project degree assignment process

  • each student is a process
  • each assignment is a process
  • administration process, knows if students fulfil criteria and marks
  • statistical information process

All supervised.

Example application: https://gitlab.com/lauramcastro/sorted

Could have gone with imperative approach.

Pros:

  • increased reliability
  • distributed responsibility
  • concurrent execution

Cons:

  • harder to control
  • harder to exaplin
  • harder to test

Take aways

  • paradigms do affect the way you think
  • strength and weaknesses
  • respect the paradigm
  • not everything is a nail

Gleam: Lean BEAM typing machine

By Louis Pilfold

BEAM and Erlang created to support multiple phone calls at once, failsafe system and perform a great number of tasks at the same time.

Erlang is a unit of concurrency. Erlang is a unit, how many tasks can be handled per second.

Named after Agner Erlang.

Each thread / process handles a single unit, and can be sequential.

"It is difficult to write a web server to handle 2 million session. But it's easy to write 1 server, and replicate it 2 million times" - Joe

No shared state between process, but instead communicates by sending messages between each other.

Distributed computing features built into the BEAM, process can send messages between each other between the same computer or a cluster of computers.

If something goes wrong, the error is contained in the smallest possible sub-system: an erlang process.

In this sense, BEAM is similar to Kubernetes, although no shared state and more granular when it comes to fail-over.

They operate at a different level of abstraction.

The BEAM, almost by chance, became super useful for webservers, databases, queues, other systems!

Languages on the BEAM: Erlang, Elixir, Gleam and others.

The BEAM gives us a the tools to tolerate mistakes, e.g. Bugs in production!

Gleam and ML languages

Compiler as a tool, as a pair programming buddy that gives you hint what might go wrong.

Complementing the fault tolerance of the BEAM, with the compiler and static analysis of Gleam.

Helps reduces and minimize the feedback loop before going to production with an error introduced by a programmer.

Gleam tries to be a type-safe version of OTP and Erlang!

Offensive programming

If your business logic expects something to work, don't be defensive on it.

Assert that every step worked as expected and return as soon as possible if there is an error.

Let is crash!

Differentiate on errors:

  • user input
  • "expected" errors (network failures, external services down)
  • unexpected errors (Oops)

gleam.run

Cotonic: browser coroutines with an universal MQTT message bus

By Marc Worrell

MQTT - Message Queueing Telemetry Transport

A communication protocol often used in IoT

Uses topic trees, with wild-cards, can be deep as you want.

Can have anything as a payload, erlang termns, binaries etc.

Client connect to server broker (over Web Sockets), over a bridge.

Example at cotonic.org/examples/chat

assets/images/posts/beam-v/mqtt-cotonic.png

Security

We need ACL, and not everything on the same bus.

Privacy!

Through a client-id and routing-id (can be seen as public IP).

Matching routing-id replies only to public and response topic.

Every access and message is authenticated (+ACL) through and Auth Cookie in Zotonic/Cotonic.

Some payloads are very private: password, chats etc.

-> Encryption

Key server, handshake to secure trust.

It has a table with communication keys for each client.

The client requests a key-id.

Encryption/decryption through a key-id.

An update from the OTP team

blog.erlang.org/OTP-23-Highlights/

WIP

Useful resources

Authenticated uplinks with verdaccio

Found a typo? Edit this page on GitHub

Written on   2020-05-26

137 words - 1 minute πŸ•œ

Context

Let's say you managed to set up a private verdaccio npm registry.

Previously you were publishing (scoped) packages on npmjs.com (that required authentication).

Of course you didn't republish all the old packages to your new verdaccio installation.

Verdaccio authenticated uplinks

Verdaccio has the concept of uplinks, that can also be authenticated via a Token.

To use your new verdaccio registry and fallback to npmjs.com registry, you can achieve that with the follow configuration in your ~/.config/verdaccio/config.yaml:

uplinks:
  npmjs:
    url: https://registry.npmjs.org/
    auth:
      type: bearer
      token: "YOUR_NPM_TOKEN"

This way, packages and package versions that do not exist on your new verdaccio instance, are being proxied over from npm!

Additionally, this means that you can now use verdaccio as the only registry in your .npmrc!


logo from https://verdaccio.org/docs/en/logo

Testing in Node.js by example using the SOLID principles

Found a typo? Edit this page on GitHub

Written on   2020-05-20

948 words - 7 minutes πŸ•œ

An example application

I'm going to outline an example to make the concept clearer:

You need to send a recurring email to some users (a newsletter for example), based on some business logic.

The business logic / user story could say:

"As a user

I want to receive an email every week

So that I can stay up to date with the latest news."

This is intentionally a contrived example to reduce the scope of the exercise.

Topics, Approaches and Tools

Concepts discussed below are

  • SOLID principles
    • Single Responsibility Principle
    • Open-Closed Principle
    • Dependency Inversion Principle
  • Reason for a software component to change
  • Test doubles like Spies and Stubs
  • Collaborators in tests
  • User Acceptance tests
  • Unit tests

On the technical side, I am going to use the following tools

  • sinon for easy Test Doubles
  • ava for running the tests
  • MongoDB as an example, but of course the persistence can be changed at your liking
  • no real email API service, to keep it simple

A testable implementation

Test-driven design

It's important to come up with clear collaborators and responsibilities for your internal modules and functions.

This comes almost for free if you start your application by test-driving it using TDD.

The design emerges and your tests will scream if they need a collaborator

A failing user acceptance test

To start, let's imagine to have a user that signed up in the past.

The user didn't yet receive the newsletter via email.

test('sends a newsletter to users that did not yet receive it', async t => {
  await UserRepository.insert({ name: 'test', email: '[email protected]', lastEmailSentAt: null })

  await newsletter.run()

  // what should I assert here??
  // we need a collaborator for the newsletter...
})

As you can see I have difficulties to define what to assert / test.

Define the collaborators

To define the collaborators I try to follow the Single Responsibility Principle (from the SOLID principles family).

Meaning that a software component (class etc.) should have a single reason to change.

You can take it a step further and see it from the business point of view:

Who is the "actor" (like a person or business sector) associated to a specific software component or module?

That person is the reason for a software component to change.

Practical examples?

  • Marketing could want to change the frequency of the newsletter
  • Marketing wants to change the contents of the newsletter
  • You or DBA's want to save the users in a different place

You could continue on here, but let's stay practical and let the design emerge.

Unit-testing the UserRepository

Below you can see a unit-test for the repository.

It asserts that when calling the method findNotYetReceivedNewsletter, the correct number of users are returned.

It verifies that the underlying collection is queried correctly.

test('find users that did not yet receive the newsletter', async t => {
  await UsersCollection.insert({ name: 'test', email: '[email protected]', lastEmailSentAt: new Date() })
  await UsersCollection.insert({ name: 'test', email: '[email protected]', lastEmailSentAt: null })

  sinon.spy(UsersCollection, 'find')

  const users = await userRepository.findNotYetReceivedNewsletter()
  t.true(Array.isArray(users))
  t.is(users.length, 1)

  t.true(UsersCollection.find.calledOnce)
  t.true(UsersCollection.find.calledWith({ lastEmailSentAt: null }))
})

Unit-testing the EmailService

In this case, the actual sending of the emails is "stubbed" out, so no emails are actually sent.

I verify that the send method of the email module is actually called.

test('sends newsletter to user', async t => {
  const user = { name: 'test', email: '[email protected]', lastEmailSentAt: null }
  const emailStub = sinon.stub(email, 'send')
  await emailService.sendTo(user)
  t.is(emailStub.callCount, 1)
})

Inverting dependencies with the DIP principle

One could be inclined to put all the logic inside a function and call it a day.

It could work, but for how long? Or better: how do you effectively test it?

One approach could be to inject those dependencies and collaborators in a controlled manner.

We're effectively inverting the dependencies by avoiding depending on details, but abstractions.

E.g.

In the first UAT I think it's apparent that we need to provide the newsletter module at least a way to retrieve the users.

Let's try it with a UserRepository, and assert that the Users collection has been queried for users that did not receive a newsletter yet:

test('sends a newsletter to users that did not yet receive it', async t => {
  await db.get('users').insert({ name: 'test', email: '[email protected]', lastEmailSentAt: null })
  sinon.spy(userRepository, 'findNotYetReceivedNewsletter')

  await newsletter.run(userRepository)

  t.is(userRepository.findNotYetReceivedNewsletter.callCount, 1)
  // we still need a collaborator for sending the newsletter...
})

Using .spy since I want the DB to be queried, and later assert that the function has been called.

For the email, I don't want the real function to be called, hence using .stub.

I am passing in the collaborators in the tests, and use the real implementations in the application code.

We're making the newsletter code open for extension, but closed for modification

async function run (userRepository, emailService) {
  const users = await userRepository.findNotYetReceivedNewsletter()
  for (const user of users) {
    await emailService.sendTo(user)
  }
}

Stubbing the email service

test('sends a newsletter to users that did not yet receive it', async t => {
  await db.get('users').insert({ name: 'test', email: '[email protected]', lastEmailSentAt: null })

  sinon.spy(userRepository, 'findNotYetReceivedNewsletter')
  sinon.stub(emailService, 'sendTo')

  await newsletter.run(userRepository, emailService)

  t.is(userRepository.findNotYetReceivedNewsletter.callCount, 1)
  t.is(emailService.sendTo.callCount, 1)
  // implement your assertions about arguments, like "recipient", "subject", "content" of the email etc.
})

The emailService could interact with the Mailgun API, Sendgrid, etc. That's up to you.

File structure

Below an outline of the file structure of the project:

β”œβ”€β”€ index.js
β”œβ”€β”€ index.test.js
β”œβ”€β”€ lib
β”‚   β”œβ”€β”€ db.js
β”‚   β”œβ”€β”€ email-service.js
β”‚   β”œβ”€β”€ email-service.test.js
β”‚   β”œβ”€β”€ email.js
β”‚   β”œβ”€β”€ email.test.js
β”‚   β”œβ”€β”€ user-repository.js
β”‚   └── user-repository.test.js
β”œβ”€β”€ package-lock.json
└── package.json

The other software components are Unit tested, for further details check out the project on GitHub

Every collaborator has their own tests (except lib/db.js since it's a simple wrapper around monk that is already well tested).

email.js is the wrapper around your API of choice to send emails and has a single exposed function .send.

You can find the full project on GitHub.

Clean up Mac OS: How I freed 35GB of space

Found a typo? Edit this page on GitHub

Written on   2020-05-20

235 words - 1 minute πŸ•œ

My disk space was running low. I had less than 100MB of free space. I was sick of running low on disk space.

Clean My Mac wasn't an option. I don't want to spend 40$ for a cleaning tool, if you're so inclined feel free to do it.


A search on GitHub revealed the following scripts (not very much up to date, but they get their job done):

rm -rf all the things

I selectively ran the steps in clean_my_mac.sh.

The available disk space went up a few GB.

https://media.giphy.com/media/Vd317WWBZKB6zP1WV1/giphy.gif

Ran a few steps from maid.sh which helped to free up some more GB:

...
#Taking out the trash.
printf "Emptying the trash.\n"
sudo rm -rfv /Volumes/*/.Trashes > /dev/null 2>&1
sudo rm -rfv ~/.Trash  > /dev/null 2>&1

#Clean the logs.
printf "Emptying the system log files.\n"
sudo rm -rfv /private/var/log/*  > /dev/null 2>&1
sudo rm -rfv /Library/Logs/DiagnosticReports/* > /dev/null 2>&1

printf "Deleting the quicklook files.\n"
sudo rm -rf /private/var/folders/ > /dev/null 2>&1
...

mac-cleanup

Then I installed mac-cleanup with sh -c "$(curl -fsSL https://raw.githubusercontent.com/fwartner/mac-cleanup/master/installer.sh)" and ran cleanup -n.

It's gonna take a while.

-n skips brew. brew is going to be inspected separately.

inspecting brew

Running brew cask list list revealed a whole lot of heavy casks installed in the past. nuke them.

Simply get the name of the cask and run brew cask uninstall [CASK_NAME].

Did the same for brew list, then running brew uninstall [NAME].

mac-cleanup again

Ran cleanup (this time without the -n flag), and this is the result:

mac cleaned

Fixing 431 Request Header Fields Too Large in Node.js

Found a typo? Edit this page on GitHub

Written on   2020-05-18

256 words - 2 minutes πŸ•œ

You're seeing a blank page saying "HTTP_ERROR 431"?

And you're running a Node.js HTTP server, like express or fastify?

Running node --help states:

  ...

  --max-http-header-size=...                set the maximum size of HTTP headers (default: 8KB)

  ...

This Node.js CLI flag can help:

--max-http-header-size=16384

It sets the HTTP Max Headers Size to 16KB.

This is due to a "recent" (November 2018) change in Node.js.

Namely a fix for a discovered vulnerability "Denial of Service with large HTTP headers (CVE-2018-12121)".

The Fix says:

All versions of 6 and later are vulnerable and the severity is HIGH.

By using a combination of many requests with maximum sized headers (almost 80 KB per connection), and carefully timed completion of the headers, it is possible to cause the HTTP server to abort from heap allocation failure.

Attack potential is mitigated by the use of a load balancer or other proxy layer.

The gist is that

The total size of HTTP headers received by Node.js now must not exceed 8192 bytes.

In other words if your query string, cookies sent along with your HTTP requests exceed 8KB, Node.js is replying to the client with the error above.

It wasn't immediate to find the solution to this, and I actually would advise to not increase the limit if absolutely necessary and you have valid motivations.

The problem is most likely with the management of cookies on the client side, excessive query strings or a combination of both.

The fix for the CVE makes total sense, thus: know what you're doing.

Notes on "Code BEAM V 2020" Authenticated uplinks with verdaccio Testing in Node.js by example using the SOLID principles Clean up Mac OS: How I freed 35GB of space Fixing 431 Request Header Fields Too Large in Node.js visit the /archive to see all 137 posts Setting up a Verdaccio npm registry Privacy and Coherence Elixir trick: start an Observer window with mix Validate your RSS feed Minimal dark mode with CSS and JavaScript Using Β΅compress to dramatically optimize static assets Ad blocking with Raspberry Pi and Pi-hole Optional chaining in Node.js 14 Nullish coalescing in Node.js 14 Road to Elixir: Monitor Crypto assets visit the /archive to see all 137 posts Resuming Elixir by self-hosting plausible analytics devblog: yet another static site generator, seriously Boring software development Full list of Chromium Puppeteer flags and command line switches How to connect puppeteer to a Proxy