Table of Contents

About

An simple URL shortener RESTful web service API written in Go, using the net/http package for the server, the HttpRouter for the router and Bolt DB for the database.

The Stoplight service was used to design and document the REST API. See below for the API Documentation

The API service is deployed on Heroku, and you can access it on url-shortener-thanoskoutr.herokuapp.com.

Because the Heroku Free plan is no longer available from November 28, 2022, the above link does not work. I wish to deploy the API in a Heroku free alternative service in the future.

The purpose of this project was to get familiar with Go, and create a backend service (a REST API specifically), to get to know the basic packages and web constructs. The focus was only on the backend, thus an API spec and documentation was created, even though we have very few endpoints. In the future, a UI could be build (e.g. with React) to create a full stack application.

GitHub GitHub repository: https://github.com/thanoskoutr/url-shortener

Techonologies

The main technologies used for this project (language, tools, database):

golang logo boltdb logo stoplight logo heroku logo

Usage

Run service

To run the web service:

$ git clone https://github.com/thanoskoutr/url-shortener.git
$ cd url-shortener
$ go run main.go

Starting the server on :8080

Build executable

To build and run the web service as an executable:

$ go build
$ ./url-shortener

Starting the server on :8080

Call the API

Shorten a URL:

$ curl -X POST http://localhost:8080/shorten -d '{"long_url":"https://www.github.com"}'

{"long_url":"https://www.github.com","short_url":"GJWrQa0"}

Access the URL you shortened from the following new URL:

$ curl http://localhost:8080/redirect/GJWrQa0

<a href="https://www.github.com">Moved Permanently</a>.

Testing

To run the tests for all subpackages (recursively):

$ go test -v ./...

Dependencies

To fetch the dependencies from go.mod:

$ go mod download

To create the go.mod from scratch (only for dev purposes), run:

$ go mod init github.com/thanoskoutr/url-shortener
$ go get github.com/boltdb/bolt
$ go get github.com/julienschmidt/httprouter
$ go get github.com/rs/cors
$ go mod tidy

Deployment

For instruction on how to deploy this Go web service (or any other Go app) for free on a Platform as a Service cloud provider like Heroku, see the Heroku Deployment Guide.

Features

  • Create Short URL from Long URL (/shorten endpoint)
    • Get a Long URL from user, as a JSON body parameter
    • Use encoding algorithm (MD5 Hash and base64 encoding)
    • Save (Short URL, Long URL) key-value pair in Database
    • Return Short URL as a JSON response to the user
  • Redirect server (/redirect/:short_url endpoint)
    • If :short_url parameter is not empty, redirect to saved Long URL on Database
    • If :short_url parameter is empty, redirect to Home page (/)

API Specification

You can find the API Documentation on Spotlight.

Contains:

  • API Documentation.
  • Ability to make calls to the API Endpoints of the service.

BoltDB Web Interface

We can inspect the BoltDB database in the browser using the boltdbweb tool.

To install it globally (usually in the ~/go/bin folder in your system), run:

go install github.com/evnix/boltdbweb@latest

Usage:

boltdbweb -d urls.db -p 8081
  • --db-name: The file name of the DB.
    • NOTE: If file.db does not exist. it will be created as a BoltDB file.
  • --port: Port for listening on… (Default: 8080)

The BoltDB Web Interface wil look something like this:

BoltDB Web Interface Example Screenshot

URL Shortening Techniques

In this section, we analyze some basic URL shortening techniques and what we have chosed to implement.

Bases

  • base58: [1-9]][a-k][m-z][A-H][J-N][P-Z]
  • base62: [0–9][a-z][A-Z]
  • base64: [0–9][a-z][A-Z][+/]
  • base64: [0–9][a-z][A-Z][-_] (URL and Filename safe)
123456789ABCDEFGH JKLMN PQRSTUVWXYZabcdefghijk mnopqrstuvwxyz
= 58 characters = base58

0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
= 62 characters = base62

0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz+/
= 64 characters = base64

0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-_
= 64 characters = base64 (URL and Filename safe)
base62

Number of possible short URLs using base62 and No of characters:

  • 6 characters short URL: 62^6 = ~56.8 billion possible URLs
  • 7 characters short URL: 62^7 = ~3.52 trillion possible URLs
  • 8 characters short URL: 62^8 = ~218 trillion possible URLs
base64

Number of possible short URLs using base64 and No of characters:

  • 6 characters short URL: 64^6 = ~68.7 billion possible URLs
  • 7 characters short URL: 64^7 = ~4.4 trillion possible URLs
  • 8 characters short URL: 64^8 = ~281 trillion possible URLs

Techniques

  • Short URL from random numbers
  • Short URL from hashing (MD5, SHA1, SHA256)
  • Short URLs from base conversion (from base10 to base62)
  • Key Generation Service (KGS)

Implementated Technique

Source Code in shortener/shortener.go, function Encode:

  • Add scheme to URL if not provided (http://, https://)
  • Get the MD5 Hash of the URL
  • Encode the Hash to Base 64 (URL and Filename safe)
  • Keep the first 7 characters of the encoded string