ChrisAs soon as Elixir was released a few years ago, I got into it and started learning and building things with it.
Thanks to the help of some dear colleagues I had the opportunity to get valuable feedback and learn even more concepts about GenServers, the BEAM virtual machine, ETS and Erlang+Elixir in general.
I also went to my first Erlang+Elixir conf, and had the chance & honour to meet Joe Armstrong, #rememberingjoe.
On 2020/04/22, during quarantine, I decided to get back to Elixir (who knows: maybe even dabble with Erlang directly).
I post-poned this too much now, it’s time to get back to the distributed programming world.
Without further ado, below my journey (in form of a daily journal) about learning more about Elixir (again), the BEAM, plausible analytics and things discovered along the way!
2020/04/23
Forked the repo -> christian-fei/plausible
2020/04/24
Updated to latest elixir version with brew upgrade elixir.
Was on 1.9.1, 1.10.2 iss the latest stable release at the time of writing.
Watched “Elixir: A Mini-Documentary 2018”, very interesting video about the Elixir’s history, touching on topics from the BEAM virtual machine, IoT scalability and general usage on the web services that can handle million of connections.
2020/04/25
Started following these accounts on Twitter:
Then this happened
Self hosting and license
Seems all sorted out right away.
The repository on Github states:
Can Plausible be self-hosted?
At the moment we don’t provide support for easily self-hosting the code. Currently, the purpose of keeping the code open-source is to be transparent with the community about how we collect and process data.
And about the license:
Plausible is open-source under the most permissive MIT license. There are no restrictions on redistributing, modifying or using this software for any reason.
Let’s go then!
Getting started with the plausible elixir code
git clone [email protected]:christian-fei/plausible.git
cd plausibleDownload deps
mix deps.getRun the tests
mix testThis yields the error about the missing postgresql connection
09:11:52.095 [error] GenServer #PID<0.6248.0> terminating
** (DBConnection.ConnectionError) tcp connect (localhost:5432): connection refused - :econnrefused
(db_connection) lib/db_connection/connection.ex:87: DBConnection.Connection.connect/2
(connection) lib/connection.ex:622: Connection.enter_connect/5
(stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: nil
** (Mix) The database for Plausible.Repo couldn't be created: killedSo,let’s boot up a postgresql instance with docker:
docker run -d -p 5432:5432 -v postgres-data:/var/lib/postgresql/data --name postgres1 postgresNow all the tests run fine, as expected:
~/D/p/plausible (master) mix test
.........................................................................................................................................................................................................................
Finished in 5.6 seconds
217 tests, 0 failures
Randomized with seed 396356Running locally with mix phx.server
mix phx.serverGot the error
[error] Postgrex.Protocol (#PID<0.5924.0>) failed to connect: ** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name) database "plausible_dev" does not existSolved with mix ecto.create:
~/D/p/plausible (master) mix ecto
Ecto v3.4.2
A toolkit for data mapping and language integrated query for Elixir.
Available tasks:
mix ecto.create # Creates the repository storage
mix ecto.drop # Drops the repository storage
mix ecto.dump # Dumps the repository database structure
mix ecto.gen.migration # Generates a new migration for the repo
mix ecto.gen.repo # Generates a new repository
mix ecto.load # Loads previously dumped database structure
mix ecto.migrate # Runs the repository migrations
mix ecto.migrations # Displays the repository migration status
mix ecto.reset # Alias defined in mix.exs
mix ecto.rollback # Rolls back the repository migrations
mix ecto.setup # Alias defined in mix.exs
~/D/p/plausible (master) mix ecto.create
The database for Plausible.Repo has been createdRun the migrations:
~/D/p/plausible (master) mix ecto.migrate
11:16:39.587 [info] == Running 20181201181549 Plausible.Repo.Migrations.AddPageviews.change/0 forward
11:16:39.589 [info] create table pageviews
11:16:39.604 [info] == Migrated 20181201181549 in 0.0s
11:16:39.636 [info] == Running 20181214201821 Plausible.Repo.Migrations.AddNewVisitorToPageviews.change/0 forward
11:16:39.636 [info] alter table pageviews
11:16:39.638 [info] == Migrated 20181214201821 in 0.0s
...
Almost there:
~/D/p/plausible (master) mix phx.server
[info] Running PlausibleWeb.Endpoint with cowboy 2.7.0 at 0.0.0.0:8000 (http)
[error] Could not start node watcher because script "/Users/christian/Documents/projects/plausible/assets/node_modules/webpack/bin/webpack.js" does not exist. Your Phoenix application is still running, however assets won't be compiled. You may fix this by running "cd assets && npm install".
[info] Access PlausibleWeb.Endpoint at http://localhost:8000I had to compile the assets with npm install, after that, everything seems fine:
~/D/p/plausible (master) mix phx.server
[info] Running PlausibleWeb.Endpoint with cowboy 2.7.0 at 0.0.0.0:8000 (http)
[info] Access PlausibleWeb.Endpoint at http://localhost:8000
webpack is watching the files…Uuuuuuuh! It’s working!
Registering 127.0.0.1:8080
Included the following script on my blog to set up local tracking:
<script async defer data-domain="127.0.0.1" src="http://localhost:8000/js/plausible.js"></script>Clicked the activation link printed in the console, instead of the email (since the email wasn’t sent).
While I set up the local tracking for 127.0.0.1:8080, I noticed that the website wasn’t registering.
It kept showing “Waiting for first pageview on 127.0.0.1”
So, looking through the source, I noticed that in the files p.js and plausible.js, there was a guard to ignore local tracking:
if (/^localhost$|^127(?:\.[0-9]+){0,2}\.[0-9]+$|^(?:0*\:)*?:?0*1$/.test(window.location.hostname)) return ignore('website is running locally');Commented this line, and my local site is finally set up!
removing concept of “trial” from the code
In lib/plausible_web/templates/layout/app.html.eex you can find the part that shows the remaining trial days.
It’s an eex file (like .erb in Ruby), which stands for Embedded Elixir. Used as templates in Elixir for short. More info here..
<%= if @conn.assigns[:current_user].subscription == nil do %>
<li class="mr-6 hidden sm:block">
<%= link(trial_notificaton(@conn.assigns[:current_user]), to: "/settings") %>
</li>
<% else %>
<li class="mr-6 hidden sm:block">
<%= link("Give feedback", to: "/feedback") %>
</li>
<% end %>So, a user has a subscription. I want this subscription to be valid and “active” forever.
Faking the trial_expiry_date
Looking through the code, Plausible.Repo, Plausible.Auth.User and Plausible.Billing.Subscription seem interesting files to dig deeper into.
Load your user, in an Elixir Interactive Shell iex.
Run iex -S mix:
user = Plausible.Repo.one(Plausible.Auth.User)
# we got the user (there is only one locally..)Importing Ecto.Changeset allows you to change given properties on an Ecto model:
import Ecto.ChangesetLet’s give our lucky user 100 years of free trial:
This returns an Ecto changeset, that we’ll later use to update the user model:
changeset = Plausible.Repo.one(Plausible.Auth.User) |> change(trial_expiry_date: Timex.today() |> Timex.shift(years: 100))
#Ecto.Changeset<
action: nil,
changes: %{trial_expiry_date: ~D[2120-04-25]},
errors: [],
data: #Plausible.Auth.User<>,
valid?: true
>Great. Now update the user through Plausible.Repo.update!:
Plausible.Repo.update!(changeset)And the result is the following:
UPDATE "users" SET "trial_expiry_date" = $1, "updated_at" = $2 WHERE "id" = $3 [~D[2120-04-25], ~N[2020-04-25 14:51:39], 1]
%Plausible.Auth.User{
__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
email: "[REDACTED]",
google_auth: #Ecto.Association.NotLoaded<association :google_auth is not loaded>,
id: 1,
inserted_at: ~N[2020-04-25 09:48:16],
last_seen: ~N[2020-04-25 14:10:00],
name: "Christian Fei",
password: nil,
password_hash: "$2b$12$U1QBbtTh/4JAsCYuHdrCfeg.uMQGZwEbMWlmNWXPryKgdOgJBKosS",
site_memberships: #Ecto.Association.NotLoaded<association :site_memberships is not loaded>,
sites: #Ecto.Association.NotLoaded<association :sites is not loaded>,
subscription: #Ecto.Association.NotLoaded<association :subscription is not loaded>,
trial_expiry_date: ~D[2120-04-25],
updated_at: ~N[2020-04-25 14:51:39]
}| From 30 days of trial | we managed to “extend” it til the year 2120 |
 |  |
Nice.
2020/04/26
Trying to run plausible with Docker.
Stumbled upon bitwalker/alpine-elixir-phoenix which seems like a nice (and up to date) docker image for phoenix projects.
Dockerfile
Added a Dockerfile for plausible and it looks like this:
FROM bitwalker/alpine-elixir-phoenix:latest
EXPOSE 8000
ADD . .
RUN mix do deps.get, deps.compile
ADD assets/package.json assets/
RUN cd assets && \
npm install
RUN cd assets/ && \
npm run deploy && \
cd - && \
mix do compile, phx.digest
USER root
ENTRYPOINT ["/opt/app/run.sh"]Where run.sh sets up Ecto and starts phoenix:
#!/bin/sh
cd /opt/app
mix ecto.create
mix ecto.migrate
mix phx.serverTo stitch everything together, this is the docker-compose.yml I came up:
version: "3"
volumes:
node_modules:
build:
services:
postgres:
image: postgres:11-alpine
restart: always
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
web:
build: .
restart: always
environment:
- MIX_ENV=docker
- PORT=8000
ports:
- "80:8000"
depends_on:
- postgresThe application runs in the environment docker, which is similar to dev, except for the Plausible.Repo hostname, that is set to postgres.
This because the web container “knows” only the postgres host, that is resolved to the container and lets phoenix connect to postgres in the Docker environment.
Deployment with docker-compose
Easy peasy. Set up a Droplet with DigitalOcean, the smallest one for 5$ is well more than fine..
Followed this guide to a basic installation of docker and docker-compose.
Read about how to always restart the docker containers (also at boot) with --restart always.
Had to change the BASE_URL when building the static assets, to point to plausible.cri.dev.
Sbam. plausible.cri.dev is running behind Cloudflare with SSL, in a docker containers with docker-compose, in the smallest DigitalOcean droplet.
PS: don’t even try to out smart it and sign up to it, no signup email is sent so you won’t be able to access the dashboard.
Additionally, had to add this snippet to this very site:
<script async defer data-domain="cri.dev" src="https://plausible.cri.dev/js/plausible.js"></script>
Ah, and then I set my personal user’s trial expiration date to 100 years in the future.
I hope that’s enough :)
Next up?
Probably it’s best to disable signups, have to dig deeper in the code (currently no mail is sent, so no new users can sign up)
Remove the concept of subscription and trial, further investigation needed.
Set up a Google Client ID and Secret to get search keywords through the google console
Same for Twitter
Update 2020-05-06
A kind person on GitHub forked the repo christian-fei/plausible and showed me how to do this even better.
Better in the sense:
Instead of signup up a user, and then manually modifying its trial_expiry_date, he simply changed the code so that by default any new user would have an extended trial!
The diff looks like this for lib/plausible/auth/user.ex:
- |> change(trial_expiry_date: Timex.today() |> Timex.shift(days: 30))
+ |> change(trial_expiry_date: Timex.today() |> Timex.shift(years: 100))Anyways, I had the chance to dabble a bit with Phoenix and Ecto, so not so bad after all.