Commerce of the future

Category: Typescript Development

  • Why bother learning golang?

    Why bother learning golang?

    As a solution architect, tech lead, lead developer or developer you are often faced by a task at hand to solve and it is natural to solve it using the tools that you are comfortable using. In most cases, the better you are familiar with your tools, the solution will be implemented in a good way and the problem solved with few issues once deployed to production. This works most of the times, but not always!

    One about the great things about Moores Law is that the what ever problem we were not able to solve yesterday, we will be able to solve tomorrow due to the ever increasing hardware made available to run more CPU cycles and doing more work per cycle than ever before.

    Today, you could do almost anything by implementing a full stack solution based on a modern Javascript framework and a type safe language on top of it such as Typescript. Even if Javascript in the nodejs environment has it’s limitations, the tools at hand can make those limitations go away by just adding more computing power and memory.

    Is Javascript a slow and really CPU-intensive language where you are not able to run blazing fast applications? The answer must be “It depends”. Every C# or golang coder will come up with graphs and solutions to show what nodejs is not the way to go even for small programs.

    The race

    The best way to find out is probably to try to perform the same task by the same program written in two different languages. Let’s go for the so often JSON-parsing of data where the volume of the data to parse is 10Mb and to do it 100 times for reading and writing data 100 times.

    Before running the test, I was pretty sure that golang would outperform Javascript every day of the week and I gave Javscript the advantage of upgrading the node environment to version 21 and golang using version 1.20.

    This is the source code for index.js

    The first surprise and what I was not expecting was that when I did the setting of the dataSize to be 10Mb, then I got a time for the nodejs version of the program running in less than 10 seconds on my Macbook Pro Intel. 

    const fs = require('fs');
const crypto = require('crypto');

// Burn some CPU while generating JSON data
function generateRandomData(sizeInBytes) {
    const buffer = crypto.randomBytes(sizeInBytes);
    return JSON.stringify({ data: buffer.toString('base64') });
}

const filePath = './test.json';
const dataSize = 1024 * 1024 * 10; // 10MB
const iterations = 100;

async function writeFile() {
    const data = generateRandomData(dataSize);
    await fs.promises.writeFile(filePath, data);
}

async function readFile() {
    await fs.promises.readFile(filePath, 'utf8');
}

async function main() {
    console.time('Node.js');
    for (let i = 0; i < iterations; i++) {
        await writeFile();
        await readFile();
    }
    console.timeEnd('Node.js');
}

main().catch(err => console.error(err));

    I run it several times to get a goot output.

    fgn@fredrikmbp javascript % time node index.js 
Node.js: 11.100s
node index.js  8.28s user 2.53s system 94% cpu 11.416 total
fgn@fredrikmbp javascript % time node index.js
Node.js: 9.889s
node index.js  7.95s user 2.30s system 102% cpu 9.999 total
fgn@fredrikmbp javascript % time node index.js
Node.js: 9.626s
node index.js  7.80s user 2.28s system 103% cpu 9.720 total
fgn@fredrikmbp javascript % time node index.js
Node.js: 9.637s
node index.js  7.90s user 2.31s system 104% cpu 9.746 total

    I was not expecting this to be so fast, so I wrote the same program using golang which is supposed to be a really fast and efficient language. 

    package main

import (
	"crypto/rand"
	"encoding/base64"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"time"
)

type Data struct {
	Data string `json:"data"`
}

// Burn some CPU while generating JSON data
func generateRandomData(sizeInBytes int) (Data, error) {
	buffer := make([]byte, sizeInBytes)
	_, err := rand.Read(buffer)
	if err != nil {
		return Data{}, err
	}
	return Data{Data: base64.StdEncoding.EncodeToString(buffer)}, nil
}

const filePath = "test.json"
const dataSize = 1024 * 1024 * 10 // 10MB
const iterations = 100

func writeFile(data Data) error {
	jsonData, err := json.Marshal(data)
	if err != nil {
		return err
	}
	return ioutil.WriteFile(filePath, jsonData, 0644)
}

func readFile() (Data, error) {
	file, err := os.Open(filePath)
	if err != nil {
		return Data{}, err
	}
	defer file.Close()

	var data Data
	err = json.NewDecoder(file).Decode(&data)
	return data, err
}

func main() {
	start := time.Now()
	for i := 0; i < iterations; i++ {
		data, err := generateRandomData(dataSize)
		if err != nil {
			log.Fatal(err)
		}
		if err = writeFile(data); err != nil {
			log.Fatal(err)
		}
		if _, err = readFile(); err != nil {
			log.Fatal(err)
		}
	}
	fmt.Printf("Go: %v\n", time.Since(start))
}

    Now let’s see how golang is performing. This code is not written in any specific way where it is optimized for the target environment.

    fgn@fredrikmbp javascript % time go run test.go
Go: 16.5503746s
go run test.go  11.99s user 6.30s system 100% cpu 18.141 total
fgn@fredrikmbp javascript % time go run test.go
Go: 16.427243349s
go run test.go  11.96s user 6.09s system 107% cpu 16.776 total
fgn@fredrikmbp javascript % time go run test.go
Go: 16.482000925s
go run test.go  11.97s user 6.09s system 107% cpu 16.817 total
fgn@fredrikmbp javascript % time go run test.go
Go: 16.395016073s
go run test.go  11.94s user 6.07s system 107% cpu 16.739 total

    My program developed using golang was performing well completing the task in 16,5 seconds. It’s really predictible and writing and then reading 10Mb JSON-data 100 times is really an ok response time to get it done in 16 seconds, but wait!! This is not what I expected. The node program is running faster than my compiled binary golang program.

    Why would I use golang?

    Now speed is complex and for now even with Just in Time (JIT) compilers, languages and tools such as nodejs and PHP are catching up on speed but still there are advantages with a language where the output is a binary file created by a compiler. It is also in the structure of how the environment handles multiple threads and what kind of locking that happens between threads.

    There are other benefits with golang for building a backend solution that I think is worth mentioning and why I’m often using Go as my backend choice.

    • The built in standard library in Golang is pretty competent and not so many additional modules are required as external dependencies. With fewer external dependencies, it is easier to keep the application up to date without the requirements to have a complex set up of libraries with versions that need to match.
    • Widely used application architectures for how to structure the applications.
    • There are a couple of frameworks that stand out for making it easier to be a developer. My favourites are:
      • Gin Web Framework https://gin-gonic.com/ which adds the missing parts of the standard library to building proper applications.
      • Fiber https://gofiber.io/ where the version 2 has been there for a long time and the version 3 is on it’s way and that hopefully soon will be released.
    • Type safety which nowdays with Typescript is not really a good reason, but since it is happening build time a lot of the potential problems are solved as long as you avoid the dynamic structs in the general case.
    • Several options for working with databases, either to select to go through one of the ORMs such as gorm https://gorm.io/ but also good options when working directly with SQL.

    I would select golang when the project is going to be larger and a bit more complex and I will require a rock solid infrastructure with few dependencies and when there are CPU- or IO-intense applications that need to be maintained over time. The ability to write tests even for API routes is really good thanks to the framework featuers in Gin.

    When is Nodejs and Typescript the right choice?

    If I would have asked myself this question two years ago, I would have claimed that even if there are good frameworks for server side rendering and a way to build a typescript application with a backend I would still not build a backend in Nodejs. This is no longer true and I often build backends using Nodejs and Typescript. There are excellent choices to move forward.

    I would however not build without a framework that will keep track of the dependencies so that the work well together. Going library mode where I would select only the libraries of my choosing is really easy to get lost in the upgrade nightmare.

    Hono, https://hono.dev/ is one of the great options for how to build a web application with a backend. It is based on standards and allows for type safety over RPC operations. This is an excellent choicie if you wish to deploy your backend i.e. as Cloudflare workers.

    Nextjs, https://nextjs.org is now rather easy to deploy and run outside of Vercel if you would choose to do so. It has a steeper learning curve and has been undergoing a lot of structural changes to become the framework it is today.

    So for a small solution only requiring a basic backend with a database and an ORM (I’m often using Prisma https://www.prisma.io/) since it gives me a really easy and type safe way to interact with a database and one of the frameworks for running the application.

    I would go this way for a smaller application where time is of the essence and the logic is not that complex. With only a couple of methods and objects, it is amazing how fast you can build a full stack application based on this technology and not all applications need to have support for thousands of concurrent users.

    Bun is going to change it all

    You are not bound to only use nodejs as the runtime environment, there is this option that I’ve been trying out for some time https://bun.com/ which claims to be 100% nodejs compatible. With the speed I’m seeing in my tests with bun, I’m definitly going to considering now once it has come past version 1.0 (currently version 1.2).

    So when do I use what?

    Did this article not provide a valid answer to which solution you should choose? I guess that it is because the correct answer depends on what you wish to do, who is going to do it and how much support you would have from AI-tools for building it.

    All of the options presented here are really good options for a backend infrastructure.

    So I will end this post by just leaving the choice to you.

  • Getting started with Typescript and Express

    Getting started with Typescript and Express

    There are many tutorials in how to get started with Typescript and Express, but I still wish to contribute with my own favorite for how to set up a project even if it’s a microservice or a bit larger piece of code. To fully understand my setup, I will need to publish a couple of posts so here we go.

    Why Typescript?

    As a developer for many years, I like to be able to discover any problems early and to be able to use a type safe code even when building a small project that would be easy to create just in plain Javascript. After discovering Typescript, it allows me to better interoperate with types from different languages in the world of REST interfaces.

    Typescript does a great job in discovering problems before they happen by compiling and the output is still understandable as javascript code.

    What about express?

    The express server is really quick and neat to set up and start using. It is fast and does its job really good. It works well in a Docker environment, but is more or less still required to have a proxy in front of it in a production environment.

    Getting started

    First, you need to have npm to be able to install and run the nodejs infrastructure. Once you download nodejs, npm is included. It is the only native package that you need to install, even if I recommend using an editor such as Visual Studio Code.

    Once nodejs is installed, you can create your first project. We will be using the command line to build this.

    First we need to create a new application using a package. This is done by running the command “npm init -y” to init the application with default settings. You can of course edit them later.

    fgn@Fredriks-MBP:typescript-project$ npm init -y
Wrote to /Users/fgn/dev/typescript-project/package.json:

{
  "name": "typescript-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

    Now you need to install typescript. You need to install it globally to support the typescript command to run on your machine.

    Issue the command to install typescript on your computer and in the project. The sudo is sometimes required to gain access to writing the files. 

    sudo npm install typescript -g

    If you already have typescript installed globally, you just need to install it as a package.

    npm install typescript

    Then, to initialize the configuration file for Typescript, you could now use the command to create a new tsconfig.json file with comments and some default options selected. It will be created in the current directory.

    tsc --init

    Now open your favourite editor and open the file tsconfig.json. Edit the three settings:

    “sourceMap”:true,
    “outDir”: “./build”,
    “rootDir”: “./src”,

    This is to have a structure for sources and other files separated.

    Now you can add the additional packages to your project before we start writing the code. This will install the types and the package express so that we could get an endpoint up and running.

    npm install express @types/express 

    The code for the project

    The code could be implemented in just one file in this case, but for the upcoming blog posts, we will add some middleware etc to the project.

    Create the file src/server.ts that will contain the code for the express server.

    mkdir src
code .

    Now we will create a simple service that will return just the status of 200 and return a text string of OK when requesting it using HTTP and port 4000.

    First we will go through the service row by row and then you will find the complete code below (even if it’s really short).

    import express = require('express')

    This will import the express server package so that we could refer to it as express in the code.

    const app : express.Application = express();
    const portNumber: number = 4000;

    This will create a new constant, the app that is typed as an express.Application and then assigned to be a new express() server by invoking express() as a method. We also assign the port number here in a constant. Later on, we would get the port number from configuration.

    app.get('/', (req, res) => {
  res.send('OK').sendStatus(200);
});

    Register a path and the verb “GET” for the request with path /. Once a request hits the service, this method will be invoked in case of path /. It will send back the result to the res variable, which is bound to the result.

    In this case, it will return the string “OK” and the status of 200 OK.

    app.listen(portNumber, () => {
  console.log(`Listening to port ${portNumber}`);
});

    Now it’s time to register the listener of the port configured above. Note the quotes that will allow us to refer to variables within the string as a ${variable} notation.

    The complete code is here.

    import express = require('express')

const app : express.Application = express();
const portNumber: number = 4000;

app.get('/', (req, res) => {
    res.send('OK').sendStatus(200);
});

app.listen(portNumber, () => {
    console.log(`Listening to port ${portNumber}`);
});

    Now it’s time to make the code compile by adding a build task to the project. Open your package.json and add the row to build the project using typescript compiler tsc.

    Set the “main” to “build/server.js”, remove the “test” from scripts and add the “start”:”tsc && node ./build/server.js” instead. This is since build/server.js will be the primary file to run the project. Running the start script will later on in the blog post show how to build and start the server.

    {
  "name": "typescript-project",
  "version": "1.0.0",
  "description": "",
  "main": "build/server.js",
  "scripts": {
    "start": "tsc && node ./build/server.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
   "@types/express": "^4.17.6",
    "express": "^4.17.1",
    "typescript": "^3.9.5"
  }
}

    Now run the command to build the project.

    npm run start

    This will build the file build/server.js which is what is required to be able to run the server. You can then run the server and build it at the same time using the start command.

    Testing the service using either a simple tool such as curl or postman will give the result of our service

    curl --verbose --get http://localhost:4000

    Watching directory for changes

    One great feature of programming languages that are not compiles is that once a file is changed, the project is reloaded and it just continues with no extra restarts of the scripts.

    This is of course available also when using Typescript by the package ts-node-dev which which do just this. So back to our terminal and install another package and the ts-node to run typescript.

    npm install ts-node ts-node-dev

    To support running this command instead of the one above, just create a new script in the package.json named “dev” which will run the typescript in development mode watching any changes to any source file with a starting poinf of our server.ts.

    “dev”:”ts-node-dev ./src/server.ts”

    {
  "name": "typescript-project",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "dev": "ts-node-dev ./src/server.ts",
    "start": "tsc && node ./build/server.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@types/express": "^4.17.6",
    "express": "^4.17.1",
    "ts-node": "^8.10.2",
    "typescript": "^3.9.5"
  }
}

    Now you will be able to run your server and watch for any changes made where it will restart automatically.

    npm run dev

    Now a change of the source files would result in a restart.

    This is all for now and I do hope that you have been able to set up your first Typescript-Express piece of code. In the next post, we will look into middlewares.