Web Services (with integrated web server)

The services of a cloud-native solution are providing their functionality through web API(s). We implement this application layer with a light weight web server (to be used as start-up main entry point) of the web servie(s). This web server could be responsible for:

  • Backend service API(s)
  • Frontend API(s) to support any administration or user self-service UI
  • Static web resources used with the UI

Service project structure

+--- module-A
   +---src            ; source code (sub-projects)
   +---tst            ; unit test code (projects)
+--- module-B
   +--- ...
+---server            ; web API(s)
   +---rsc            ; extra resources like api documentation
   +---src
   |       api.server.proj
   |       appsettings.json
   +---tst
   \---ui             *** ui code

Local Development Environment

For running a local instance of the service/application, a database system for data-store persistence is required. Depending on the configuration this requires

  • a Micosoft SQL Server Express (localdb) that can be downloaded from: Download Center You could verifiy from a command line if you already have an localdb instance installed: sqllocaldb info (If there are no instances listed, create a new instance by SqlLocalDB create v11.0)
  • a PostgreSQL installation

Starting and publishing the server

To start the server in a local dev. environment:

Change into service/src/Tlabs.xxxService and run >dotnet run

To publish the server into a self-contained folder:

dotnet publish -r win-x64 -o bin/deploy (Typical runtime identifier (RIDs) are:

  • Windows portable
    win-x64, win-x86
  • Windows 10 / Server 2016
    win10-x64
  • Linux portable (Most desktop distributions like CentOS, Debian, Fedora, Ubuntu and derivatives)
    linux-x64
  • Red Hat Enterprise Linux
    rhel-x64
  • macOS
    osx-x64

Server and Configuration

The web service server is based on the ASP.NET Core hosting environment, server and middleware. (Please refer to: ASP.NET Core fundamentals)

Other than mandated by the ASP.NET Core framework, we do not rely entirely on the hardcoded 'fluent style' configuration, but use a more flexible configuration scheme heavily driven by entries in the appsettings.json.

appsettings.js Structure

The configuration of the server application is specified in the appsettings.json file in the service/server project.

The current main sections of the appsettings.json are:

  • logging
    This section is configuring a Microsoft.Extensions.Logging ILoggerFactory with Serilog as file logging provider.
    See for: General logging

  • webHosting
    configures the hosting environment.

  • configurator (see below for the general layout of a configurator)
    This configures any hosting service(s) - especially the actual web-server (implementation) to handle HTTP requests.

  • applicationMiddleware
    This configures the ASP.NET Core middleware request processing pipeline.

NOTE: Since the order of middleware components in the processing pipeline is important, entries getting ordered in memory using the 'ord:' property...

Configurator

Configurators (i.e. classes implementing IConfigurator<T> are used for a more dynamic configuration (in contrast to the hardcoded 'fluent style' configuration mandated from ASP.NET Core.)
Configurators are getting dynamically loaded and instatiated to add their configuration to a given configuration 'builder' like a IServiceCollection or IApplicationBuilder. For any components that should be put in place (configured) this way, they need to provide their own IConfigurator<T> implementation.

  • Configurator (discriptor) properties
    The Configurator section typically consists of an arbitrary hash-key like "ASPNET.MVC": in "applicationServices": { and the descriptor properties:

    • type (mandatory)
      The assembly-qualified name of the configurator class.

    • ord (optional)
      Ordinal number used to sort the configurators (if their order matters).

    • config (optional)
      An optional object hash to contain any configurator specific properties. (This requires the configurator to have a ctor taking a IDictionary<string, string> to contain these settings.)

Data seeding

The application's database is created and seeded with some sample automatically when starting the application server for the first time.

Data seeding if performed through classes that provide an implementation of the IDataSeed interface. They loaded dynamically in the applicationServices section of the appsettings.json file.

For local development SQLServer LocalDB is required to be installed.

HTTP REST API

API Goals:

  • Following web standards where it makes sense.
  • URLs should be easiliy explorable from a web browsers addressbar in order to make the API the developers UI.
  • Intuitive, simple schema for pleasant adoption.
  • Should support and fit with the majority of exisiting UI technologies with minimal need for fitting.
  • Reasonable efficient while balancing other requirements.

RESTful Resources and Actions

The REST concept results into separation of logical resources being manipulated with actions.

  • Resources A resource coresponds with a model type (technically a DTO) typically closely related with an entity (e.g. ProductDoc). In the scope of the API, resources are always named with a plural noun (api/products) to keep things simple and consistant. In certain cases resources could also be denormalized/flattened versions of more complex types where this suits the needs of a UI representation.

  • CRUD Actions The main (CRUD) actions to manipulate resources are expressed with HTTP request methods (GET, POST, PUT, PATCH, DELETE):

    • GET api/products - Retrieves a list of product(s) (Optionaly filtered and sorted - see below)

    • GET api/products/5 - Retrieves a specific product with Id: 5

    • POST api/products - Creates a new product

    • PUT api/products/5 - Updates product #5

    • DELETE api/products/5 - Deletes product #5

  • Special Actions For situations where the desired action does not fit in the CRUD pattern, any special actions are treated like sub-resources:

    • GET api/tasks/5/done - Mark the specific task #5 as 'done'

Result filtering & sorting

Any optional parameters for filterting and sorting are passed with query parameters where the parameter contents are in JSON format.

GET api/products?filter=[{"property":"lastname","value":"xyz","operator":"like"}]&sorter=[{"property":"fieldName","direction":"ASC"}]

Request Body

The data transmitted with POST or PUT requests must be in JSON format. The Content-Type header should reflect this format accordingly. All methods for resource modification are returning the entire resource with their response. (See below.) This is because there might happen modifications to resource properties that are not under the control of the client like: Id or Creation(date) on POST...

Response

The response of all APIs is in JSON format wrapped into an envelope/cover:

{
  success: true,
  error: "failure description", //only present if success == false
  total: 123, // optional
  data: {}  //actual resource or array of resources
}

Even if some might discourage the use of envelopes it is believed that the extra flexibillity in passing extensible metadata (e.g. for paging) and also the default support by many UI frameworks justifies its use. It is preferred to always return pretty print formatted JSON for better readabillty during development and compensate the extra data transfer with gzip support.

Errors and HTTP Status Codes

Basically errors are falling into this two categories:

  • Client Errors Any errors caused by the client in passing invalid request data that can not be processed by the server. The response is still being a JSON envelope/cover with the error property giving any details about the failure. The HTTP Status has to be in the 400 range:

    • 400 Bad Request - The request is malformed, such as if the body does not parse
    • 401 Unauthorized - When no or invalid authentication details are provided.
    • 403 Forbidden - When authentication succeeded but the authenticated user doesn't have access to the resource
    • 404 Not Found - When a non-existent resource is requested
    • 405 Method Not Allowed - When an HTTP method is being requested that isn't allowed for the authenticated user
    • 410 Gone - Indicates that the resource at this end point is no longer available. Useful as a blanket response for old API versions
    • 415 Unsupported Media Type - If incorrect content type was provided as part of the request
    • 422 Unprocessable Entity - Used for validation errors
    • 429 Too Many Requests - When a request is rejected due to rate limiting

  • Server Errors Typically there are limitted chances to return a proper JSON cover. If this is not possible the response should be text/plane and should give some exception trace.

  • In the success case these status codes should be applied:

    • 200 OK - Response to a successful GET, PUT, or DELETE. Can also be used for a POST that doesn't result in a creation.
    • 201 Created - Response to a POST that results in a creation. Maybe combined with a Location header pointing to the location of the new resource
    • 204 No Content - Response to a successful request that won't be returning a body (typically it is prepfered to always return a cover)
    • 304 Not Modified - Used when HTTP caching headers are in play

Authentication

In general authentication requires some token to be transmitted with each API request. The prefered place where to technically put this token could depend on the actual client technology in use. As a compromize to support most client types this would be a pragmatic approach:

  1. First the API looks in the Authorization header for the authentication data it needs, since that's probably the place where non-browser clients will prefer to put it. If the authentication data is placed here, it must be in the format of a ApiKey token. For example:

     GET /resource HTTP/1.1
 Host: server.example.com
 Authorization: ApiKey mF_9.B5f-4.1JqM

  2. To provide simplifyed and streamlined support for browser-based clients, the API also checks for a (session?) cookie for server side log in, in case when the regular Authorization header was missing.

DTO Model

  • Model types define the representation of the API resources. In most cases this model representation would be very close to the applications entity types and thus makes it tempting to just use the entities directly. But in order to decouple the API from internal domain logic and to make it more stable even when refactoring entites, it should be worth the extra effort to introduce extra model types as API resource representations.

  • Mapping entities to model objects. Among the extra effort of model types introduction there is the need to convert (or map) back and forth between entities and API model types.

    Even though there are tools like AutoMapper promising to reduce the extensive convertion code, their configuration effort for the more interesting cases (property name or value convertion, bidirectional mapping, flattening) seems to level the amount of code neccessary plus the extra need to get familiar with those.

    So, just sticking with plain ctor and convertion methods like AsEntity() in the model type should be the way to go.