Logging and Configuration

We provide our customers with a lot of flexibility in logging. In this context, you can have your logs logged in any one or more of the following environments at the same time.

Console
Elastic Search
File
Seq

1- Console

You don't need a 3rd party software to print your logs to the console. You can make your settings only through the environment variable.

{
    "Serilog": {
        "WriteTo": [
            {
                "Name": "Console",
                "Args": {
                  "outputTheme": "Literate",
                  "outputTemplate": "[{Level} {@Timestamp:yyyy-MM-dd HH:mm:ss}] [Message: {Message} {Exception}]{NewLine}",
                  "restrictedToMinimumLevel": "Information"
                }
            }
        ]
    }
}

"Name": This key specifies where the logs will be written. The "Console" value indicates that the logs will be written to the console.
"Args": This key defines the arguments required for the specified target. Detailed explanations are given below.
- "outputTheme": This key specifies the theme of the console output. "Literate" is used here, which means that the logs will be formatted in a more readable way. Possible values for this key are:
  - Literate
  - Colored
  - Grayscale
  - None
- "outputTemplate": This key defines the format of the log messages. The provided template specifies how each log message will be structured. In this case, it includes:
  - {Level}: The severity level of the log (e.g., Information, Warning, Error).
  - {@Timestamp}: The timestamp of when the log was created, formatted as "YYYY-MM-DD HH:MM:SS".
  - {Message}: The actual log message.
  - {Exception}: Any exception details if applicable.
  - {NewLine}: A new line character to separate log entries.

2- Elasticsearch

Prerequisites

Docker

If you have not installed Docker, download and install the version of Docker that is appropriate for your operating system. You can find installation instructions on Docker's official website.

2.1 Download and Install Elasticsearch

you can install elastic search and kibana using this yml file. docker-compose.yml

For more customizations of Elasticsearch, you can check out the official documentation of Elasticsearch.

2.2 Log Configuration

Now, we con go over Elasticsearch configuration for our project.

{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Elasticsearch",
        "Args": {
          "nodeUris": "{Your Elasticsearch node URI comes here}",
          "basicAuthUsername": "{Your Elasticsearch username comes here}",
          "basicAuthPassword": "{Your Elasticsearch password comes here}",
          "indexFormat": "apilogs-{logType}-{0:yyyy.MM.dd}",
          "autoRegisterTemplate": true,
          "autoRegisterTemplateVersion": "ESv8",
          "emitEventFailure": "WriteToSelfLog"
        }
      }
    ]
  }
}

"Name": This key specifies where the logs will be written. The "Elasticsearch" value indicates that the logs will be written to Elasticsearch.
"Args": This key defines the arguments required for the specified target. Detailed explanations are given below.
- "nodeUris": This key specifies the URI of the Elasticsearch node where logs will be sent. You should replace "{Your Elasticsearch node URI comes here}" with the actual URI of your Elasticsearch node (e.g., "http://localhost:9200").
- "basicAuthUsername": This key is used to specify the username for basic authentication with Elasticsearch. Replace "{Your Elasticsearch username comes here}" with your actual username.
- "basicAuthPassword": This key is used to specify the password for basic authentication with Elasticsearch. Replace "{Your Elasticsearch password comes here}" with your actual password.
- "indexFormat": This key specifies the format of the index where logs will be stored in Elasticsearch. The value "apilogs-{logType}-{0:yyyy.MM.dd}" indicates that logs will be stored in an index named "apilogs" followed by the log type and date (e.g., "apilogs-info-2023.10.01"). if "{logType}" is not specified, all logs will not be categorized by type.
- "autoRegisterTemplate": This key indicates whether to automatically register an index template in Elasticsearch. Setting it to true means that Serilog will create a template for the specified index format.
- "autoRegisterTemplateVersion": This key specifies the version of the index template to register. In this case, it is set to "ESv8", indicating compatibility with Elasticsearch version 8.
- "emitEventFailure": This key specifies how to handle failures when emitting log events to Elasticsearch. Setting it to "WriteToSelfLog" means that any failures will be logged to Serilog's self-log, allowing you to diagnose issues with logging.

3- File

{
    "Serilog": {
        "WriteTo": [
            {
              "Name": "File",
              "Args": {
                "path": "logs/{logType}-log-.txt",
                "outputTemplate": "[{Level} {@Timestamp:yyyy-MM-dd HH:mm:ss}] [Message: {Message} {Exception}]{NewLine}",
                "rollingInterval": "Day"
              }
            }
        ]
    }
}

"Name": This key specifies where the logs will be written. The "File" value indicates that the logs will be written to a file.
"Args": This key defines the arguments required for the specified target. Detailed explanations are given below.
- "path": This key specifies the path where the log files will be stored. The value "logs/{logType}-log-.txt" indicates that logs will be stored in a "logs" directory, and the file name will include a placeholder for the log type (e.g., "info-log-2023-10-01.txt"). if "{logType}" is not specified, all logs will not be categorized by type.
- "outputTemplate": This key defines the format of the log messages. The provided template specifies how each log message will be structured. In this case, it includes:
  - {Level}: The severity level of the log (e.g., Information, Warning, Error).
  - {@Timestamp}: The timestamp of when the log was created, formatted as "YYYY-MM-DD HH:MM:SS".
  - {Message}: The actual log message.
  - {Exception}: Any exception details if applicable.
  - {NewLine}: A new line character to separate log entries.
- "rollingInterval": This key specifies how often a new log file should be created. In this case, it is set to "Day," meaning a new log file will be created every day.Possible values for this key are:
  - Year
  - Month
  - Day
  - Hour
  - Minute

4- SEQ

Prerequisites

Docker

If you have not installed Docker, download and install the version of Docker that is appropriate for your operating system. You can find installation instructions on Docker's official website.

4.1 Download and Install SEQ

Download the latest version of Seq by typing the following command in the terminal or command client:

docker pull datalust/seq:latest

Start the Seq container with the following command. This command starts Seq with default settings:

docker run -d \
  --name seq \
  -p 5341:5341 \
  -v /path/to/your/data:/data \
  -e SEQ_API_KEY=your_api_key_here \
  datalust/seq:latest

* -d: It allows the container to run in the background. * --name seq: Sets the container name to "seq". * -p 5341:5341: Redirects port 5341 on your local machine to port 5341 of the Seq container. You will use this port to access Seq from your browser. * -v /path/to/your/data:/data Replace "/path/to/your/data" with your own file path. This way, Seq data will be stored in the specified local directory. * -e SEQ_API_KEY=your_api_key_here: (Optional) Used to set the API key. This may be preferred to increase security.

Once the container is running, open your browser and visit the following URL:

```bash http://localhost:5341

Here you can access Seq's web interface and view logs. For further customization of Seq, you can check out [Seq's official documentation](https://docs.datalust.co/docs/an-overview-of-seq).


### 4.2 Log Configuration

Now, we con go over seq configuration for our project.

```json 
{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Seq",
        "Args": {
          "serverUrl": "{Your Seq server URL comes here}",
          "apiKey": "{Your Seq API Key comes here}"
        }
      }
    ]
  }
}

"Name": This key specifies where the logs will be written. The "Seq" value indicates that the logs will be written to Seq.
"Args": This key defines the arguments required for the specified target. Detailed explanations are given below.
- "serverUrl": This key specifies the URL of the Seq server where logs will be sent. You should replace "{Your Seq server URL comes here}" with the actual URL of your Seq server (e.g., "http://localhost:5341").
- "apiKey": This key is used to specify the API key for authentication with the Seq server. Replace "{Your Seq API Key comes here}" with your actual API key if you have set one up in your Seq instance.

A sample configuration

{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTheme": "Literate",
          "outputTemplate": "[{Level} {@Timestamp:yyyy-MM-dd HH:mm:ss}] [Message: {Message} {Exception}]{NewLine}"
        }
      },
      {
        "Name": "Elasticsearch",
        "Args": {
          "nodeUris": "{Your Elasticsearch node URI comes here}",
          "basicAuthUsername": "{Your Elasticsearch username comes here}",
          "basicAuthPassword": "{Your Elasticsearch password comes here}",
          "indexFormat": "apilogs-{logType}-{0:yyyy.MM.dd}",
          "autoRegisterTemplate": true,
          "autoRegisterTemplateVersion": "ESv8",
          "emitEventFailure": "WriteToSelfLog"
        }
      },
      {
        "Name": "File",
        "Args": {
          "path": "logs/{logType}-log-.txt",
          "outputTemplate": "[{Level} {@Timestamp:yyyy-MM-dd HH:mm:ss}] [Message: {Message} {Exception}]{NewLine}",
          "rollingInterval": "Day"
        }
      },
      {
        "Name": "Seq",
        "Args": {
          "serverUrl": "{Your Seq server URL comes here}",
          "apiKey": "{Your Seq API Key comes here}"
        }
      }
    ],
    "Properties": {
      "Application": "Pointr.Microservice.Content.API",
      "AllowedMethods": ""
    }
  }
}

"Properties": This key allows you to add custom properties to your logs. In this example, it includes:
- "Application": Specifies the name of the application generating the logs (e.g., "Pointr.Microservice.Content.API").
- "AllowedMethods": This can be used to specify allowed HTTP methods. (e.g., "GET, POST, PUT, DELETE"). This property can be dynamically set based on your application's requirements. If empty, it will consider to be all methods allowed.

This configuration allows you to log messages to the console, a file, Seq, and Elasticsearch simultaneously, providing flexibility in how you manage and analyze your logs.