Skip to main content

4 posts tagged with "middleware"

View All Tags

Ported to Cloud with Winglang (Part One)

· 35 min read
Asher Sterkin
Technical Writer

Blue Zone Application from “Hexagonal Architecture Explained

Fig 1: The “Blue Zone” Application Ported to Cloud with Wing

Directly porting software applications to the cloud often results in inefficient and hard-to-maintain code. However, using the new cloud-oriented programming language Wing in combination with Hexagonal Architecture has proven to be a winning combination. This approach strikes the right balance between cost, performance, flexibility, and security.

In this series, I will share my experiences migrating various applications from mainstream programming languages to Winglang. My first experience implementing the Hexagonal Architecture in Wing was reported in the article "Hello, Winglang Hexagon!”. While it was enough to acquire confidence in this combination, it was built on an oversimplified "Hello, World" greeting service, and as such lacked some essential ingredients and was insufficient to prove the ability of such an approach to work at scale.

In Part One, I focus on porting the “Blue Zone” application, featured in the recently published book “Hexagonal Architecture Explained”, from Java to Wing. The “Blue Zone” application brings in a substantial code base, still not too huge to dive into unmanageable complexity, yet representative of a large class of applications. Also, the fact that it was originally written in mainstream Java brings an interesting case study of creating a cloud-native variant of such applications.

This report also serves as a tribute to Juan Manuel Garrido de Paz, the book's co-author, who sadly passed away in April 2024.

Before we proceed, let's recap the fundamentals of the Hexagonal Architecture pattern.

The Hexagonal Architecture Pattern Essentials

Refer to Chapter Two of the “Hexagonal Architecture Explained” book for a detailed and formal pattern description. Here, I will bring an abridged recap of the main sense of the pattern in my own words.

The Hexagonal Architecture pattern suggests a simple yet practical approach to separate concerns in software. Why is the separation of concerns important? Because the software code base quickly grows even for a modest in terms of delivered value application. There are too many things to take care of. Preserving cognitive control requires a high-level organization in groups or categories. To confront this challenge the Hexagonal Architecture pattern suggests splitting all elements involved in a particular software application into five distinct categories and dealing with each one separately:

  1. The Application itself. This category encapsulates the real value delivered to prospective customers and users. This is the reason why software is going to be developed and used in the first place. Sometimes, it’s called the Core or System Under Development (SuD). Another possible name for this part could be Computation - where external inputs are processed and final results are produced. Visually, the Application part of the system is represented in the form of a hexagon. There is nothing special or magic in this shape. As the “Hexagonal Architecture Explained” book authors explain:

“Hexagonal Architecture” has served well as a hook to the pattern. It’s easy to remember and generates conversation. However, in this book we want to be correct: The name of the pattern is “Ports & Adapters”, because there really are ports, and there really are adapters, and your architecture will show them.

  1. External Actors that communicate with or are communicated by the Application. These could be human end users, electronic devices, or other Applications. The original pattern suggests further separation into Primary (or Driving) Actors - those who initiate an interaction with the Application, and Secondary (or Driven) Actors - those with whom the Application initiates communication.

  2. Ports - a fancy name for formal specification of Interfaces the Primary Actors could use (aka Driving Ports) or Secondary Actors need to implement (aka Driven Ports) to communicate with the Application. In addition to the formal specification of the interface verbs (e.g. BuyParkingTicket) Ports also provide detailed specifications of data structures that are exchanged through these interfaces.

  3. Adapters fill the gaps between External Actors and Ports. As the name suggests, Adapters are not supposed to perform any meaningful computations, but rather basically convert data from/to formats the Actors understand to/from data ****the Application understands.

  4. Configurator pulls everything together by connecting External Actors to the Application through Ports using corresponding Adapters. Depending on the architectural decisions made and price/performance/flexibility requirements these decisions were trying to address, a specific Configuration can be produced statically before the Application deployment or dynamically during the Application run.

Contrary to popular belief, the pattern does not imply that one category, e.g. Application, is more important than others, nor does it suggest ultimately that one should be larger while others smaller. Without Ports and Adapters, no Application could be practically used. Relative sizes are often determined by non-functional requirements such as scalability, performance, cost, availability, and security.

The pattern suggests reducing complexity and risk by focusing on one problem at a time, temporally ignoring other aspects. It also suggests a practical way to ensure the existence of multiple configurations of the same computation each one addressing some specific needs be it test automation or operation in different environments.

The picture below from “Hexagonal Architecture Explained” book nicely summarizes all main elements of the pattern:

Fig 2: The Hexagonal Architecture Patterns in a Nutshell

“Blue Zone” Sample Application

From the application README:

BlueZone allows car drivers to pay remotely for parking cars at regulated zones in a city, instead of paying with coins using parking meters.

  1. Driving actors using the application are car drivers and *parking inspectors.
  1. Car drivers will access the application using a Web UI (User Interface), and they can do the following:
  • Ask for the available rates in the city, in order to choose the one of the zone they want to park the car at.
  • Buy a ticket for parking the car during a period of time at a regulated zone. This period starts at current date-time. The ending date-time is calculated from the paid amount, according to the rate (euros/hour) of the zone.
  1. Parking inspectors will access the application using a terminal with a CLI (Command Line Interface), and they can do the following:
  • Check a car for issuing a fine, in case that the car is illegally parked at a zone. This will happen if there is no active ticket for the car and the rate of the zone. A ticket is active if current date-time is between the starting and ending date-time of the ticket period.
  1. Driven actors needed by the application are:
  • Repository with the data (rates and tickets) used in the application. It also has a sequence for getting ticket codes as they are needed.
  • Payment service that allows the car driver to buy tickets using a card. Obviously, no adapter for a real service has been developed, just a test-double (mock).
  • Date-time service for obtaining the current date-time when needed, for buying a ticket and for checking a car.

I chose this application for two primary reasons. First, it was recommended by the “Hexagonal Architecture Explained” book as a canonical example. Second, it was originally developed in Java. I was curious to see what is involved in porting a non-trivial Java application to the cloud using the Wing programming language.

Where Do You Start From?

The “Hexagonal Architecture Explained” book provides reasonable recommendations in Chapter 4.9, “What is development sequence?”. It makes sense to start with “Test-to-Test” and proceed further. However, I did what most software engineers normally do— starting with translating the Java code to Wing. Within a couple of part-time days, I reached a stage where I had something working locally in Wing with all external interfaces simulated.

While technically it worked, the resulting code was far too big relative to the size of the application, hard to understand even for me, aesthetically unappealing, and completely non-Wingish. Then, I embarked on a two-week refactoring cycle, looking for the most idiomatic expression of the core pattern ideas adapted to the Wing language and cloud environment specifics.

What comes next is different from how I worked. It was a long series of chaotic back-and-forth movements with large portions of code produced, evaluated, and scrapped. This usually happens in software development when dealing with unfamiliar technology and domains.

Finally, I’ve come up with something that hopefully could be gradually codified into a more structured and systematic process so that it will be less painful and more productive the next time. Therefore, I will present my findings in the conceptually desirable sequence to be used next time, rather than how it happened in reality.

Thou Shalt Start with Tests

To be more accurate, the best and most cost-effective way is to start with a series of acceptance tests for the system's architecturally essential use cases. Chapter 5.1 of the “Hexagonal Architecture Explained” book, titled “How does this relate to use cases?”, elaborates on the deep connection between use case modeling and Hexagonal Architecture. It’s worth reading carefully.

Even the previous statement wasn’t 100% accurate. We are supposed to start with identifying Primary External Actors and their most characteristic ways of interacting with the system. In the case of the “Blue Zone” application, there are two Primary External Actors:

  1. Card Driver
  2. Parking Inspector

For the Car Drive actor, her primary use case would be “Buy Ticket”; for the Parking Inspector, his primary use case would be “Check Car”. By elaborating on these use cases’ implementation we will identify Secondary External Actors and the rest of the elements.

The preliminary use case model resulting from this analysis is presented below:

Fig 3: “Blues Zone” Application Use Case Mode

Notice that the diagram above contains only one Secondary Actor - the Payment Service and does not include any internal Secondary Actors such as a database. While these technology elements will eventually be isolated from the Application by corresponding Driven Ports they do not represent any Use Case External Actor, at least in traditional interpretation of Use Case Actors.

Specifying use case acceptance criteria before starting the development is a very effective technique to ensure system stability while performing internal restructurings. In the case of the “Blue Zone” application, the use case acceptance tests were specified in Gherkin language using the Cucumber for Java framework.

Currently, a Cucumber framework for Wing does not exist for an obvious reason - it’s a very young language. While an official Cucumber for JavaScript does exist, and there is a TypeScript Cucumber Tutorial I decided to postpone the investigation of this technology and try to reproduce a couple of tests directly in Wing.

Surprisingly, it was possible and worked fairly well, at least for my purposes. Here is an example of the Buy Ticket use case happy path acceptance test specified completely in Wing:

bring "../src" as src;
bring "./steps" as steps;

/*
Use Case: Buy Ticket
AS
a car driver
I WANT TO
a) obtain a list of available rates
b) submit a "buy a ticket" request with the selected rate
SO THAT
I can park the car without being fined
*/
let _configurator = new src.Configurator("BuyTicketFeatureTest");
let _testFixture = _configurator.getForAdministering();
let _systemUnderTest = _configurator.getForParkingCars();
let _ = new steps.BuyTicketTestSteps(_testFixture, _systemUnderTest);

test "Buy ticket for 2 hours; no error" {
/* Given */
["name", "eurosPerHour"],
["Blue", "0.80"],
["Green", "0.85"],
["Orange", "0.75"]
]);
_.next_ticket_code_is("1234567890");
_.current_datetime_is("2024/01/02 17:00");
_.no_error_occurs_while_paying();
/* When */
_.I_do_a_get_available_rates_request();
/* Then */
_.I_should_obtain_these_rates([
["name", "eurosPerHour"],
["Blue", "0.80"],
["Green", "0.85"],
["Orange", "0.75"]
]);
/* When */
_.I_submit_this_buy_ticket_request([
["carPlate", "rateName", "euros", "card"],
["6989GPJ", "Green", "1.70", "1234567890123456-123-062027"]
]);
/* Then */
_.this_pay_request_should_have_been_done([
["euros", "card"],
["1.70", "1234567890123456-123-062027"]
]);
/* And */
_.this_ticket_should_be_returned([
["ticketCode", "carPlate", "rateName", "startingDateTime", "endingDateTime", "price"],
["1234567890", "6989GPJ", "Green", "2024/01/02 17:00", "2024/01/02 19:00", "1.70"]
]);
/* And */
_.the_buy_ticket_response_should_be_the_ticket_stored_with_code("1234567890");
}

While it’s not a truly human-readable text, it’s close enough and not hard to understand. There are quite a few things to unpack here. Let’s proceed with them one by one.

The Test Structure

The test above assumes a particular project folder structure and reflects the Wing module and import conventions, which states

It's also possible to import a directory as a module. The module will contain all public types defined in the directory's files. If the directory has subdirectories, they will be available under the corresponding names.

From the first two lines, we can conclude that the project has two main folders: src where all source code is located, and test where all tests are located. Further, there is a test\steps subfolder where individual test step implementations are kept.

The next three lines allocate a preflight Configurator object and extract from it two pointers:

  1. _testFixture pointing to a preflight class responsible for the test setup
  2. _systemUnderTest which points to a Primary Port Interface intended for Car Drivers.

Within the “Buy ticket for 2 hours; no errors”, we allocate an inflight BuyTicketTestSteps object responsible for implementing individual steps. Conventionally, this object gets an almost invisible name underscore, which improves the overall test readability. This is a common technique for developing a Domain-Specific Language (DSL) embedded in a general-purpose host language.

It’s important to stress, that while it did not happen in my case, it’s fully conceivable to start the project with a simple src and test\steps folder structure and a simple test setup to drive other architectural decisions.

Of course, with no steps implemented, the test will not even pass compilation. To make progress, we need to look inside the BuyTicketTestSteps class.

Test Steps Class

The test steps class for the Buy Ticket Use Case is presented below:

bring expect;
bring "./Parser.w" as parse;
bring "./TestStepsBase.w" as base;
bring "../../src/application/ports" as ports;

pub class BuyTicketTestSteps extends base.TestStepsBase {
_systemUnderTest: ports.ForParkingCars;
inflight var _currentAvailableRates: Set<ports.Rate>;
inflight var _currentBoughtTicket: ports.Ticket?;

new(
testFixture: ports.ForAdministering,
systemUnderTest: ports.ForParkingCars
) {
super(testFixture);
this._systemUnderTest = systemUnderTest;
}

inflight new() {
this._currentBoughtTicket = nil;
this._currentAvailableRates = Set<ports.Rate>[];
}

pub inflight the_existing_rates_in_the_repository_are(
sRates: Array<Array<str>>
): void {
this.testFixture.initializeRates(parse.Rates(sRates).toArray());
}

pub inflight next_ticket_code_is(ticketCode: str): void {
this.testFixture.changeNextTicketCode(ticketCode);
}

pub inflight no_error_occurs_while_paying(): void {
this.testFixture.setPaymentError(ports.PaymentError.NONE);
}

pub inflight I_do_a_get_available_rates_request(): void {
this._currentAvailableRates = this._systemUnderTest.getAvailableRates();
}

pub inflight I_should_obtain_these_rates(sRates: Array<Array<str>>): void {
let expected = parse.Rates(sRates);
expect.equal(this._currentAvailableRates, expected);
}

pub inflight I_submit_this_buy_ticket_request(sRequest: Array<Array<str>>): void {
let request = parse.BuyRequest(sRequest);
this.setCurrentThrownException(nil);
this._currentBoughtTicket = nil;
try {
this._currentBoughtTicket = this._systemUnderTest.buyTicket(request);
} catch err {
this.setCurrentThrownException(err);
}
}

pub inflight this_ticket_should_be_returned(sTicket: Array<Array<str>>): void {
let sTicketFull = Array<Array<str>>[
sTicket.at(0).concat(["paymentId"]),
sTicket.at(1).concat([this.testFixture.getLastPayResponse()])
];
let expected = parse.Ticket(sTicketFull);
expect.equal(this._currentBoughtTicket, expected);
}

pub inflight this_pay_request_should_have_been_done(sRequest: Array<Array<str>>): void {
let expected = parse.PayRequest(sRequest);
let actual = this.testFixture.getLastPayRequest();
expect.equal(actual, expected);
}

pub inflight the_buy_ticket_response_should_be_the_ticket_stored_with_code(code: str): void {
let actual = this.testFixture.getStoredTicket(code);
expect.equal(actual, this._currentBoughtTicket);
}

pub inflight an_error_occurs_while_paying(error: str): void {
this.testFixture.setPaymentError(parse.PaymentError(error));
}

pub inflight a_PayErrorException_with_the_error_code_that_occurred_should_have_been_thrown(code: str): void {
//TODO: make it more specific
let err = this.getCurrentThrownException()!;
log(err);
expect.ok(err.contains(code));
}

pub inflight no_ticket_with_code_should_have_been_stored(code: str): void {
try {
this.testFixture.getStoredTicket(code);
expect.ok(false, "Should never get there");
} catch err {
expect.ok(err.contains("KeyError"));
}
}
}

This class is straightforward: it parses the input data, uniformly presented as Array<Array<str>>, into application-specific data structures, sends them to either testFixture or _systemUnderTest objects, keeps intermediate results, and compares expected vs actual results where appropriate.

The only specifics to pay attention to are the proper handling of preflight and inflight definitions. I’m grateful to Cristian Pallares, who helped me to make it right.

We have three additional elements with clearly delineated responsibilities:

  1. Parser - ****Responsible for converting a uniform array of string inputs to the application-specific data structures.
  2. Test Fixture - ****Responsible for backdoor communication with the system for preconditions setting and postconditions verification.
  3. System Under Test - ****Responsible for implementing the application logic.

Let’s take a closer look at each one.

Parser

The source code of the Parser module is presented below:

bring structx;
bring datetimex;
bring "../../src/application/ports" as ports;

pub class Util {
pub inflight static Rates(sRates: Array<Array<str>>): Set<ports.Rate> {
return unsafeCast(
structx.fromFieldArray(
sRates,
ports.Rate.schema()
)
);
}

pub inflight static BuyRequest(
sRequest: Array<Array<str>>
): ports.BuyTicketRequest {
let requestSet: Set<ports.BuyTicketRequest> = unsafeCast(
structx.fromFieldArray(
sRequest,
ports.BuyTicketRequest.schema()
)
);
return requestSet.toArray().at(0);
}

pub inflight static Tickets(
sTickets: Array<Array<str>>
): Set<ports.Ticket> {
return unsafeCast(
structx.fromFieldArray(
sTickets,
ports.Ticket.schema(),
datetimex.DatetimeFormat.YYYYMMDD_HHMM
)
);
}

pub inflight static Ticket(sTicket: Array<Array<str>>): ports.Ticket {
return Util.Tickets(sTicket).toArray().at(0);
}

pub inflight static PayRequest(
sRequest: Array<Array<str>>
): ports.PayRequest {
let requestSet: Set<ports.PayRequest> = unsafeCast(
structx.fromFieldArray(
sRequest,
ports.PayRequest.schema()
)
);
return requestSet.toArray().at(0);
}

pub inflight static CheckCarRequest(
sRequest: Array<Array<str>>
): ports.CheckCarRequest {
let requestSet: Set<ports.CheckCarRequest> = unsafeCast(
structx.fromFieldArray(
sRequest,
ports.CheckCarRequest.schema()
)
);
return requestSet.toArray().at(0);
}

pub inflight static CheckCarResult(
sResult: Array<Array<str>>
): ports.CheckCarResult {
let resultSet: Set<ports.CheckCarResult> = unsafeCast(
structx.fromFieldArray(
sResult, ports.CheckCarResult.schema()
)
);
return resultSet.toArray().at(0);
}

pub inflight static DateTime(dateTime: str): std.Datetime {
return datetimex.parse(
dateTime,
datetimex.DatetimeFormat.YYYYMMDD_HHMM
);
}

pub inflight static PaymentError(error: str): ports.PaymentError {
return Map<ports.PaymentError>{
"NONE" => ports.PaymentError.NONE,
"GENERIC_ERROR" => ports.PaymentError.GENERIC_ERROR,
"CARD_DECLINED" => ports.PaymentError.CARD_DECLINED
}.get(error);
}
}

This class, while not sophisticated from the algorithmic point of view, reflects some important architectural decisions with far-reaching consequences.

First, it announces a dependency on the system Ports located in the src\application\ports folder. Chapter 4.8 of the “Hexagonal Architecture Explained” book, titled “Where do I put my files?”, makes a clear statement:

The folder structure is not covered by the pattern, nor is it the same in all languages. Some languages (Java), require interface definitions. Some (Python, Ruby) don't. And some, such as Smalltalk, don't even have the concept of files!

It warns, however, that “we’ve observed that folder structures that don't match the intentions of the pattern end up causing damage”. For strongly typed languages like Java, it recommends keeping specifications of Driving and Driven Ports in separate folders.

I started with such a structure, but very soon realized that it just enlarges the size of the code and prevents it from taking full advantage of the Wing module and import conventions. Based on this I decided to keep all Ports in one dedicated folder. Considering the current size of the application, this decision looks justified.

Second, it exploits an undocumented Wing module and import feature that makes all public static inflight methods of a class named Util directly accessible by the client modules, which improves the code readability.

Third, it uses two Wing Standard Library extensions, datetimex, and structx developed to compensate for some features I needed. These extensions were part of my “In Search for Winglang Middleware” project endor.w, I reported about here, here, and here.

Justification for these extensions will be clarified when we look at the core architectural decision about representing the Port Interfaces and Data.

Representing Port Interfaces and Data

Traditional strongly typed Object-Oriented languages like Java advocate encapsulating all domain elements as objects. If I followed this advice, the Ticket object would look something like this:

pub inflight class Ticket {
pub ticketCode: str;
pub carPlate: str;
pub rateName: str;
pub startingDateTime: std.Datetime;
pub endingDateTime: std.Datetime;
pub price: num;
pub paymentId: str;

new (ticketCode: str, ...) {
this.ticketCode = ticketCode;
...
}
pub toJson(): Json {
return Json {
ticketCode = this.ticketCode,
...
}
pub static fromJson(data: Json): Ticket {
return new Ticket(
data.get("ticketCode").asStr(),
...
);
}
pub toFieldArray(): Array<str> {
return [
this.ticketCode,
...
];
}
pub static fromFieldArray(records: Array<Array<str>>): Set<Ticket> {
let result = new MutSet<Ticket>[];
for record in records {
result.add(new Ticket(
record.at(0),
...
);
}
return result.copy();
}
} such

Such an approach introduces 6 extra lines of code per data field for initialization and conversation plus some fixed overhead of method definition. This creates a significant boilerplate overhead.

Mainstream languages like Java and Python try alleviating this pain with various meta-programming automation tools, such as decorators, abstract base classes, or meta-classes.

In Wing, all this proved to be sub-optimal and unnecessary, provided minor adjustments were made to the Wing Standard Library.

Here is how the Ticket data structure can be defined:

pub struct Ticket {                 //Data structure representing objects 
//with the data of a parking ticket:
ticketCode: str; //Unique identifier of the ticket;
//It is a 10-digit number with leading zeros
//if necessary
carPlate: str; //Plate of the car that has been parked
rateName: str; //Rate name of the zone where
//the car is parked at
startingDateTime: std.Datetime; //When the parking period begins
endingDateTime: std.Datetime; //When the parking period expires
price: num; //Amount of euros paid for the ticket
paymentId: str; //Unique identifier of the payment
//made to get the ticket.
}

In Wing, structures are immutable by default, and that eliminates a lot of access control problems.

Without any change, the Wing Standard Library will support out-of-the-box Json.stringify(ticket) serialization to Json string and Ticket.fromJson(data) de-serialization. That’s not enough for the following reasons:

  1. For data storage, we need conversion of the Ticket objects to Json rather than a Json string
  2. Json serialization and de-serialization functions need to handle the std.Datetime fields correctly. Currently the Json.stringify() will convert any std.Datetime to an ISO string, but Ticket.fromJson() will fail.
  3. To support test automation and different CSV formats, we need the ability to convert data structures to and from an array of strings.
  4. There is a need for a more flexible conversion of strings to std.Datetime. For example, the “Blue Zone” application uses the YYYYMM HH:MM format.

All these additional needs were addressed in two Trusted Wing Libraries: datetimex and struct. While the implementation was not trivial and required a good understanding of how Wing and TypeScript interoperability works, it was doable with reasonable effort. Hopefully, these extensions can be included in future versions of the Wing Standard Library.

The special, unsafeCast function helped to overcome the Wing strong type checking limitations. To provide better support for actual vs expected comparison in tests, I decided that fromFieldArray(...) will return Set<...> objects. Occasionally it required toArray() conversion, but I found this affordable.

Now, let’s take a look at the main _systemUnderTest object.

ForParkingCars Port

Following the “Hexagonal Architecture Explained” book recommendations, port naming adopts the ForActorName convention. Here is how it is defined for the ParkingCar External Actor:

pub struct BuyTicketRequest { 	//Input data needed for buying a ticket 
//to park a car:
carPlate: str; //Plate of the car that has been parked
rateName: str; //Rate name of the zone where the car is parked at
euros: num; //Euros amount to be paid
card: str; //Card used for paying, in the format 'n-c-mmyyyy', where
// 'n' is the card number (16 digits)
// 'c' is the verification code (3 digits),
// 'mmyyyy' is the expiration month and year (6 digits)
}

/**
* DRIVING PORT (Provided Interface)
*/
pub inflight interface ForParkingCars {
/**
* @return A set with the existing rates for parking a car in regulated
* zones of the city.
* If no rates exist, an empty set is returned.
*/
getAvailableRates(): Set<rate.Rate>;

/**
* Pay for a ticket to park a car at a zone regulated by a rate,
* and save the ticket in the repository.
* The validity period of the ticket begins at the current date-time,
* and its duration is calculated in minutes by applying the rate,
* based on the amount of euros paid.
* @param request Input data needed for buying a ticket.
* @see BuyTicketRequest
* @return A ticket valid for parking the car at a zone regulated by the rate,
* paying the euros amount using the card.
* The ticket holds a reference to the identifier of the payment
* that was made.
* @throws BuyTicketRequestException
* If any input data in the request is not valid.
* @throws PayErrorException
* If any error occurred while paying.
*/
buyTicket (request: BuyTicketRequest): ticket.Ticket;
}

As with Ticket and Rate objects, the BuyTicketRequest object is defined as a plain Wing struct relying on the automatic conversion infrastructure described above.

The ForParkingCars is defined as the Wing interface. Unlike the original “Blue Zone” implementation, this one does not include BuyTicketRequest validation in the port specification. This was done on purpose.

While strong object encapsulation would encourage including the validate() method in the BuyTicketRequest class, with open immutable data structures like the ones adopted here, it could be done where it belongs - in the use case implementation. On the other hand, including the request validation logic in port specification brings in too many implementation details, too early.

ForAdministering Port

This one is used for providing testFixture functionality, and while it is long, it is also completely straightforward:

bring "./Rate.w" as rate;
bring "./Ticket.w" as ticket;
bring "./ForPaying.w" as forPaying;

/**
* DRIVING PORT (Provided Interface)
* For doing administration tasks like initializing, load data in the repositories,
* configuring the services used by the app, etc.
* Typically, it is used by:
* - Tests (driving actors) for setting up the test-fixture (driven actors).
* - The start-up for initializing the app.
*/
pub inflight interface ForAdministering {

/**
* Load the given rates into the data repository,
* deleting previously existing rates if any.
*/
initializeRates(newRates: Array<rate.Rate>): void;

/**
* Load the given tickets into the data repository,
* deleting previously existing tickets if any.
*/
initializeTickets(newTickets: Array<ticket.Ticket>): void;

/**
* Make the given ticket code the next to be returned when asking for it.
*/
changeNextTicketCode(newNextTicketCode: str): void;

/**
* Return the ticket stored in the repository with the given code
*/
getStoredTicket(ticketCode: str): ticket.Ticket;

/**
* Return the last request done to the "pay" method
*/
getLastPayRequest(): forPaying.PayRequest;

/**
* Return the last response returned by the "pay" method.
* It is an identifier of the payment made.
*/
getLastPayResponse(): str;

/**
* Make the probability of a payment error the "percentage" given as a parameter
*/
setPaymentError(errorCode: forPaying.PaymentError): void;

/**
* Return the code of the error that occurred when running the "pay" method
*/
getPaymentError(): forPaying.PaymentError;

/**
* Set the given date-time as the current date-time
*/
changeCurrentDateTime(newCurrentDateTime: std.Datetime): void;

}

Now, we need to dive one level deeper and look at the application logic implementation.

Implementation Details

ForParkingCarsBackend

bring "../../application/ports" as ports;
bring "../../application/usecases" as usecases;

pub class ForParkingCarsBackend impl ports.ForParkingCars {
_buyTicket: usecases.BuyTicket;
_getAvailableRates: usecases.GetAvailableRates;

new(
dataRepository: ports.ForStoringData,
paymentService: ports.ForPaying,
dateTimeService: ports.ForObtainingDateTime
) {
this._buyTicket = new usecases.BuyTicket(dataRepository, paymentService, dateTimeService);
this._getAvailableRates = new usecases.GetAvailableRates(dataRepository);
}

pub inflight getAvailableRates(): Set<ports.Rate> {
return this._getAvailableRates.apply();
}

pub inflight buyTicket(request: ports.BuyTicketRequest): ports.Ticket {
return this._buyTicket.apply(request);
}
}

This class resides in the src/outside/backend folder and provides an implementation of the ports.ForParkingCars interface that is suitable for a direct function call. As we can see, it assumes two additional Secondary Ports: ports.ForStoringData and ports.ForObtainingTime and delegates actual implementation to two Use Case implementations: BuyTicket and GetAvailableRates. The BuyTicket Use Case implementation is where the core system logic resides, so let’s look at it.

BuyTicket Use Case

bring math;
bring datetimex;
bring exception;
bring "../ports" as ports;
bring "./Verifier.w" as validate;

pub class BuyTicket {
_dataRepository: ports.ForStoringData;
_paymentService: ports.ForPaying;
_dateTimeService: ports.ForObtainingDateTime;

new(
dataRepository: ports.ForStoringData,
paymentService: ports.ForPaying,
dateTimeService: ports.ForObtainingDateTime
) {
this._dataRepository = dataRepository;
this._paymentService = paymentService;
this._dateTimeService = dateTimeService;
}

pub inflight apply(request: ports.BuyTicketRequest): ports.Ticket {
let currentDateTime = this._dateTimeService.getCurrentDateTime();
this._validateRequest(request, currentDateTime);
let paymentId = this._paymentService.pay(
euros: request.euros,
card: request.card
);
let ticket = this._buildTicket(request, paymentId, currentDateTime);
this._dataRepository.saveTicket(ticket);
return ticket;
}

inflight _validateRequest(request: ports.BuyTicketRequest, currentDateTime: std.Datetime): void {
let requestErrors = validate.BuyTicketRequest(request, currentDateTime);
if requestErrors.length > 0 {
throw exception.ValueError(
"Buy ticket request is not valid",
requestErrors
);
}
}

inflight _buildTicket(
request: ports.BuyTicketRequest,
paymentId: str,
currentDateTime: std.Datetime
): ports.Ticket {
let ticketCode = this._dataRepository.nextTicketCode();
let rate = this._dataRepository.getRateByName(request.rateName);
let endingDateTime = BuyTicket._calculateEndingDateTime(
currentDateTime,
request.euros,
rate.eurosPerHour
);
return ports.Ticket {
ticketCode: ticketCode,
carPlate: request.carPlate,
rateName: request.rateName,
startingDateTime: currentDateTime,
endingDateTime: endingDateTime,
price: request.euros,
paymentId: paymentId
};
}

/**
* minutes = (euros * minutesPerHour) / eurosPerHour
* endingDateTime = startingDateTime + minutes
*/
static inflight _calculateEndingDateTime(
startingDateTime: std.Datetime,
euros: num,
eurosPerHour: num
): std.Datetime {
let MINUTES_PER_HOUR = 60;
let minutes = math.round((MINUTES_PER_HOUR * euros) / eurosPerHour);
return datetimex.plus(startingDateTime, duration.fromMinutes(minutes));
}
}

The “Buy Ticket” Use Case implementation class resides within the src/application/usescases folder. It returns an inflight function responsible for executing the Use Case logic:

  1. Validate Request
  2. Pay for a new Ticket
  3. Create the Ticket record
  4. Store the Ticket record in the database

The main reason for implementing Use Cases as inflight functions is that all Wing event handlers are inflight functions. While direct function calls are useful for local testing, they will typically be HTTP REST or GraphQL API calls in a real deployment.

The actual validation of the BuyTicketRequest is delegated to an auxiliary Util class within the Verifier.w module. The main reason is that individual field validation might be very detailed and involve many low-level specifics, contributing little to the overall use case logic understanding.

Pulling all Components Together

Following the “Hexagonal Architecture Explained” book recommendations, this is implemented within a Configurator class as follows:

bring util;
bring endor;
bring "./outside" as outside;
bring "./application/ports" as ports;

enum ApiType {
DIRECT_CALL,
HTTP_REST
}

enum ProgramType {
UNKNOWN,
TEST,
SERVICE
}

pub class Configurator impl outside.BlueZoneApiFactory {
_apiFactory: outside.BlueZoneApiFactory;

new(name: str) {
let mockService = new outside.mock.MockDataRepository();
let programType = this._getProgramType(name);
let mode = this._getMode(programType);
let apiType = this._getApiType(programType, mode);
this._apiFactory = this._getApiFactory(
name,
mode,
apiType,
mockService,
mockService,
mockService
);
}

_getProgramType(name: str): ProgramType { //TODO: migrate to endor??
if name.endsWith("Test") {
return ProgramType.TEST;
} elif name.endsWith("Service") || name.endsWith("Application") {
return ProgramType.SERVICE;
} elif std.Node.of(this).app.isTestEnvironment {
return ProgramType.TEST;
}
return ProgramType.UNKNOWN;
}

_getMode(programType: ProgramType): endor.Mode {
if let mode = util.tryEnv("MODE") {
return Map<endor.Mode>{ //TODO Migrate this function to endor
"DEV" => endor.Mode.DEV,
"TEST" => endor.Mode.TEST,
"STAGE" => endor.Mode.STAGE,
"PROD" => endor.Mode.PROD
}.get(mode);
} elif programType == ProgramType.TEST {
return endor.Mode.TEST;
} elif programType == ProgramType.SERVICE {
return endor.Mode.STAGE;
}
return endor.Mode.DEV;
}

_getApiType(
programType: ProgramType,
mode: endor.Mode,
): ApiType {
if let apiType = util.tryEnv("API_TYPE") {
return Map<ApiType>{
"DIRECT_CALL" => ApiType.DIRECT_CALL,
"HTTP_REST" => ApiType.HTTP_REST
}.get(apiType);
} elif programType == ProgramType.SERVICE {
return ApiType.HTTP_REST;
}
let target = util.env("WING_TARGET");
if target.contains("sim") {
return ApiType.DIRECT_CALL;
}
return ApiType.HTTP_REST;
}

_getApiFactory(
name: str,
mode: endor.Mode,
apiType: ApiType,
dataService: ports.ForStoringData,
paymentService: ports.ForPaying,
dateTimeService: ports.ForObtainingDateTime
): outside.BlueZoneApiFactory {
let directCall = new outside.DirectCallApiFactory(
dataService,
paymentService,
dateTimeService
);
if apiType == ApiType.DIRECT_CALL {
return directCall;
} elif apiType == ApiType.HTTP_REST {
return new outside.HttpRestApiFactory(
name,
mode,
directCall
);
}
}

pub getForAdministering(): ports.ForAdministering {
return this._apiFactory.getForAdministering();
}

pub getForParkingCars(): ports.ForParkingCars {
return this._apiFactory.getForParkingCars();
}

pub getForIssuingFines(): ports.ForIssuingFines {
return this._apiFactory.getForIssuingFines();
}

}

This is an experimental, still not final, implementation, but it could be extended to address the production deployment needs. It adopts a static system configuration by exploiting the Wing preflight machinery.

In this implementation, a special MockDataStore object implements all three Secondary Ports: data service, paying service, and date-time service. It does not have to be this way and was created to save time during the scaffolding development.

The main responsibility of the Configuratior class is to determine which type of API should be used:

  1. Direct call
  2. Local HTTP REST
  3. Remote HTTP REST
  4. Local HTTP REST plus HTML
  5. Remote HTTP REST plus HTML

The actual API creation is delegated to corresponding ApiFactory classes.

What is remarkable about such an implementation is that the same test suite is used for all configurations, except for real HTML-based UI mode. The latter could also be achieved but would require some HTML test drivers like Selenium.

It is the first time I have achieved such a level of code reuse. As a result, I run local direct call configuration most of the time, especially when I perform code structure refactoring, with full confidence that it will run in a remote test and production environment without a change. This proves that the Wing cloud-oriented programming language and Hexagonal Architecture is truly a winning combination.

The Big Picture

Including the full source code of every module would increase this article's size too much. Access to the GitHub repository for this project is available on demand.

Instead, I will present the overall folder structure, two UML class diagrams, and a cloud resources diagram reflecting the main program elements and their relationships.

The Folder Structure

├── src
│ ├── application
│ │ ├── ports
│ │ │ ├── ForAdministering.w
│ │ │ ├── ForIssuingFines.w
│ │ │ ├── ForObtainingDateTime.w
│ │ │ ├── ForParkingCars.w
│ │ │ ├── ForPaying.w
│ │ │ ├── ForStoringData.w
│ │ │ ├── Rate.w
│ │ │ └── Ticket.w
│ │ ├── usecases
│ │ │ ├── BuyTicket.w
│ │ │ ├── CheckCar.w
│ │ │ ├── GetAvailableRates.w
│ │ │ └── Veryfier.w
│ ├── outside
│ │ ├── backend
│ │ │ ├── ForAdministeringBackend.w
│ │ │ ├── ForIssuingFinesBackend.w
│ │ │ └── ForParkingCarsBackend.w
│ │ ├── http
│ │ │ ├── html
│ │ │ │ ├── _htmlForParkingCarsFormatter.ts
│ │ │ │ └── htmlForParkingCarsFormatter.w
│ │ │ ├── json
│ │ │ │ ├── jsonForIssuingFinesFormatter.w
│ │ │ │ └── jsonForParkingCarsFormatter.w
│ │ │ ├── ForIssuingFinesClient.w
│ │ │ ├── ForIssuingFinesController.w
│ │ │ ├── ForParkingCarsClient.w
│ │ │ ├── ForParkingCarsController.w
│ │ │ └── middleware.w
│ │ ├── mock
│ │ │ └── MockDataRepository.w
│ │ ├── ApiFactory.w
│ │ ├── BlueZoneAplication.main.w
│ │ ├── DirectCallApiFactory.w
│ │ └── HttpRestApiFactory.w
│ └── Configurator.w
├── test
│ ├── steps
│ │ ├── BuyTicketTestSteps.w
│ │ ├── CheckCarTestSteps.w
│ │ ├── Parser.w
│ │ └── TestStepsBase.w
│ ├── usecase.BuyTicketTest.w
│ └── usecase.CheckCarTest.w
├── .gitignore
├── LICENSE
├── Makefile
├── README.md
├── package-lock.json
├── package.json
└── tsconfig.json

Fig 4: Folder Structure

Application logic-wise the project is small. Yet, it is already sizable to pose enough challenges for cognitive control over its structure. The current version attempts to strike a reasonable balance between multiple criteria:

  1. Maximal depth of file structure.
  2. Complexity and amount of import statements.
  3. The ratio between code that delivers intended value and code required to organize, test, and deliver it.

While computing all sets of desirable metrics is beyond the scope of this publication, one back-of-envelope calculation could be performed here and now manually: the percentage of files under the application and outside folders, including intermediate folders (let’s call “value”) and the total number of files and folders (let’s call it “stuff”). In the current version, the numbers are:

Total: 55

src/application: 16

src/: 41

Files: 43

Strict Value to Stuff Ratio: 16*100/55 = 29.09%

Extended Value to Stuff Ratio: (15+19)*100/42 = 74.55%

Is it big or little? Good or Bad? It’s hard to say at the moment. The initial impression is that the numbers are healthy, yet coming up with more founded conclusions needs additional research and experimentation. A real production system will require a significantly larger number of tests.

From the cognitive load perspective, 43 files is a large number exceeding the famous 7 +/- 2 limit of human communication channels and short memory. It requires some organization. In the current version, the maximal number of files at one level is 8 - within the limit.

The presented hierarchical diagram only partially reflects the real graph picture - cross-file dependencies resulting from the bring statement are not visible. Also, the __node_files__ folder reflecting external dependencies and having an impact on resulting package size is omitted as well.

In short, without additional investment in tooling and methodology of metrics, the picture is only partial.

We still can formulate some desirable direction: we prefer to deal as much as possible with assets generating the direct value and as little as possible with supporting stuff required to make it work. Ideally, a healthy Value to Stuff ratio would come from language and library support. Automatic code generation, including that performed by Generative AI, would reduce the typing but not the overall cognitive load.

Class Diagram

Depicting all “Blue Zone” application elements in a single UML Class Diagram would be impractical. Among other things, UML does not support directly separate representation of preflight and inflight elements. We can visualize separately the most important parts of the system. For example, here is a UML Class Diagram for the application part:

Fig 5: `src/application` Class Diagram

DO NOT PUBLISH
@startuml

left to right direction
hide members
struct Ticket
struct Rate
struct BuyTicketRequest
interface ForDrivingCars <<primary>>
ForDrivingCars ..> BuyTicketRequest
ForDrivingCars ..> Ticket
interface ForStoringData
ForStoringData ..> Rate
ForStoringData ..> Ticket
struct PayRequest
enum PaymentError
interface ForPaying
ForPaying ..> PayRequest
ForPaying ..> PaymentError
interface ForObtainingDateTime
class BuyTicket <<function>>
class Veryfier
Veryfier ..> BuyTicketRequest
BuyTicket --> Veryfier
BuyTicket ..> BuyTicketRequest
BuyTicket --> ForObtainingDateTime
BuyTicket --> ForStoringData
BuyTicket --> ForPaying
interface ForIssuingFines <<primary>>
struct CheckCarRequest
struct CheckCarResponse
ForIssuingFines ..> CheckCarRequest
ForIssuingFines ..> CheckCarResponse
class CheckCar <<function>>
CheckCar --> ForStoringData
CheckCar --> ForObtainingDateTime
CheckCar ..> CheckCarRequest
CheckCar ..> CheckCarResponse
@enduml

Notice that the IForParkingCars and ForIssuingFines primary interfaces are named differently from the Car Driver and Parking Inspector primary actors and BuyTicket and CheckCar use cases. This is not a mistake. Primary Port Interface names should reflect the Primary Actor role in a particular use case. There are no automatic rules for such a naming. Hopefully, the selected names are intuitive enough.

Notice also, that the Primary Interfaces are not directly implemented within the application module and there is a disconnect between these interfaces and use case implementations.

This is also not a mistake. The concrete connection between the Primary Interface and the corresponding use case implementation depends on configuration, as reflected in the UML Class Diagram Below:

Fig 6: Configurator Class Diagram (”Buy Ticket” Use Case only)

DO NOT PUBLISH
@startuml

hide members
interface ForDrivingCars <<primary>>
interface ForStoringData
interface ForPaying
interface ForObtainingDateTime
class BuyTicket <<function>>
BuyTicket --> ForObtainingDateTime
BuyTicket --> ForStoringData
BuyTicket --> ForPaying
class ForDrivingCarsBackEnd implements ForDrivingCars
ForDrivingCarsBackEnd --> BuyTicket
class ForDrivingCarsClient implements ForDrivingCars
class ForDrivingCarsController
ForDrivingCarsController --> ForDrivingCars
class MockDataStore implements ForStoringData, ForPaying, ForObtainingDateTime
interface IBlueZoneApiFactory
class DirectCallApiFactory implements IBlueZoneApiFactory
DirectCallApiFactory ..> ForDrivingCarsBackEnd
class HttpRestApiFactory implements IBlueZoneApiFactory
class cloud.Api
cloud.Api --> ForDrivingCarsController
HttpRestApiFactory --> cloud.Api
HttpRestApiFactory ..> ForDrivingCarsController
HttpRestApiFactory ..> ForDrivingCarsClient
HttpRestApiFactory ..> ForDrivingCarsBackend
class Configurator
Configurator --> IBlueZoneApiFactory
Configurator --> MockDataStore
@enduml

Only elements related to the “Buy Ticket” Use Case implementation and essential connections are depicted to avoid clutter.

According to the class diagram above the Configurator will decide which IBlueZoneApiFactory implementation to use: DirectApiCallFactory for local testing purposes or HttpRestApiFactory for both local and remote testing via HTTP and production deployment.

Cloud Resources

Fig 7: Cloud Resources

The cloud resources diagram presented above reflects the outcome of the Wing compilation to the AWS target platform. It is quite different from the UML Class Diagram presented above and we have to conclude that various types of diagrams complement each other. The Cloud Resources diagram is important for understanding and controlling the system's operational aspects like cost, performance, reliability, resilience, and security.

The main challenge, as with previous diagrams, is the scale. With more cloud resources, the diagram will quickly be cluttered with too many details.

The current versions of all diagrams are more like useful illustrations than formal blueprints. Striking the right balance between accuracy and comprehension is a subject for future research. I addressed this issue in one of my early publications. Probably, it’s time to come back to this research topic.

Conclusion

The experience of porting the “Blue Zone” application, featured in the recently published book “Hexagonal Architecture Explained”, from Java to Wing led to the following interim conclusions

  1. Directly porting software applications to the cloud often results in inefficient and hard-to-maintain code.
  2. Each programming language has its idiomatic way of expressing design decisions and blind translation from one to another does not work either.
  3. Implementing the Hexagonal Architecture pattern in the new cloud-oriented programming language Wing has proven to be a winning combination. This approach strikes the right balance between cost, performance, flexibility, and security.
  4. Codebase size grows fast for even a modest functionality-wise speaking application. Keeping the complexity under control requires methodology and guidelines.
  5. Graphical representations of the application logic and cloud resources are useful for illustration. Turning them into formal blueprints requires additional research.

Acknowledgments

Throughout the preparation of this publication, I utilized several key tools to enhance the draft and ensure its quality.

The initial draft was crafted with the organizational capabilities of Notion's free subscription, facilitating the structuring and development of ideas.

For grammar and spelling review, the free version of Grammarly proved useful for identifying and correcting basic errors, ensuring the readability of the text.

The enhancement of stylistic expression and the narrative coherence checks were performed using the paid version of ChatGPT 4o. The ChatGPT 4o tool was also used to develop critical portions of the Trusted Wing Libraries: datetimex and struct in TypeScript.

UML Class Diagrams were produced with the free version of the PlantText UML online tool.

Java version of the “Blue Zone” application was developed by Juan Manuel Garrido de Paz, the book’s co-author. Juan Manuel Garrido de Paz sadly passed away in April 2024. May his memory be blessed and this report serves as a tribute to him.

While all advanced tools and resources significantly contributed to the preparation process, the concepts, solutions, and final decisions presented in this article are entirely my own, for which I bear full responsibility.

Managing Winglang Libraries with AWS CodeArtifact

· 11 min read
Asher Sterkin
Technical Writer
Needs, Challenges, and Solutions

Winglang provides a solution for contributing to its Winglibs project. This is the way to go if you only need to wrap a particular cloud resource on one or more platforms. Just follow the guidelines. However, while developing the initial version of the Endor middleware framework, I had different needs.

First, the Endor library is in a very initial exploratory phase—far from a maturity level to be considered a contribution candidate for publishing in the public NPM Registry.

Second, it includes several supplementary and still immature tool libraries, such as Exceptions and Logging. These tools need to be published separately (see explanation below). Therefore, I needed a solution for managing multiple NPM Packages in one project.

Third, I wanted to explore how prospective Winglang customers will be able to manage their internal libraries.

For that goal, I decided to experiment with the AWS CodeArtifact service configured to play the role of my internal NPM Registry.

This publication is an experience report about the first phase, primarily focused on the developer’s experience with my Multi-Account, Multi-Platform, Multi-User (MAPU) environment, which I reported about here, here, and here. Specifically, I configured the AWS CodeArtifact Domain and Repository within my working account and postponed a more elaborate enterprise-grade system architecture to later stages. Let’s start with the overall solution overview.

Here is a brief description of the solution:

  1. Within my winglang account, I created an AWS CodeArtifact Domain tentatively named <organizationID>-platform.
  2. Under this Domain, I created an AWS CodeArtifact Repository tentatively named winglang-artifacts.
  3. This AWS CodeArtifact Repository is connected to the public npmjs repository, from which all third-party packages, including those from the official Winglibs, are downloaded.
  4. The AWS CodeArtifact Repository contains two types of packages:
    1. Those that were developed and published locally.
    2. Those that were cloned from the external npmjs repository.
  5. Locally developed packages belong to the @winglibs NPM Namespace. At the moment, this is a requirement determined by how the Winglang import system works.
  6. The remote EC2 desktop instance is configured to use the AWS CodeArtifact Repository as its NPM Registry using a temporary session token valid for 12 hours.
  7. As a developer, I communicate with my remote desktop using the VS Code Remote feature, described in the previous publication.

I found this arrangement suitable for a solo developer and researcher. A real organization, even of a middle size, will require some substantial adjustments — subject to further investigation.

Let’s now look at some technical implementation details.

Cloud Resources Allocation

Using Cloud Formation templates is always my preferred option. In this case, I created two simple Cloud Formation templates. One for creating an AWS CodeArtifact Domain resource:

{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a CodeArtifact Domain; to be a part of platform template",
"Resources": {
"ArtifactDomain": {
"Type" : "AWS::CodeArtifact::Domain",
"Properties" : {
"DomainName" : "o-4e7dgfcrpx-platform"
}
}
}
}

And another - for creating an AWS CodeArtifact Repository resource:

{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a CodeArtifact repository; to be a part of account template",
"Resources": {
"ArtifactRespository": {
"Type" : "AWS::CodeArtifact::Repository",
"Properties" : {
"Description" : "artifact repository for this <winglang> account",
"DomainName" : "o-4e7dgfcrpx-platform",
"RepositoryName": "winglang-artifacts",
"ExternalConnections" : [ "public:npmjs" ]
}
}
}
}

These templates are mere placeholders for future, more serious, development.

The same could be achieved with Winglang, as follows:

https://gist.github.com/eladb/5e1ddd1bd90c53d90b2195d080397381

Many thanks to Elad Ben-Israel for bringing this option to my attention. Currently, the whole MAPU system is implemented in Python and CloudFormation. Re-implementing it completely in Winglang would be a fascinating case study.

Configuring npm with the login command

I followed the official guidelines and created the following Bash script:

export CODEARTIFACT_AUTH_TOKEN=$(\
aws codeartifact get-authorization-token \
--domain o-4e7dgfcrpx-platform \
--domain-owner 851725645964 \
--query authorizationToken \
--output text)
export REPOSITORY_ENDPOINT=$(\
aws codeartifact get-repository-endpoint \
--domain o-4e7dgfcrpx-platform \
--domain-owner 851725645964 \
--repository winglang-artifacts \
--format npm \
--query repositoryEndpoint \
--output text)
export REGISTRY=$(echo "$REPOSITORY_ENDPOINT" | sed 's|https:||')
npm config set registry=$REPOSITORY_ENDPOINT
npm config set $REGISTRY:_authToken=$CODEARTIFACT_AUTH_TOKEN

Here is a brief description of the script’s logic:

  1. Using the AWS CLI, retrieve an AWS CodeArtifact session token (valid for the next 12 hours).
  2. Using the AWS CLI, the AWS CodeArtifact repository endpoint in a format compatible with NPM.
  3. Use the NPM config command to set the endpoint.
  4. Use the NPM config command to set up session authentication.

Placing this script in the [/etc/profile.d](https://www.linuxfromscratch.org/blfs/view/11.0/postlfs/profile.html) ensures that it will be automatically executed at every user login thus making the whole communication with AWS CodeArtifact instead of the official [npmjs](https://docs.npmjs.com/cli/v8/using-npm/registry) repository completely transparent for the end user.

Publishing Custom Libraries

Implementing this operation while addressing my specific needs required a more sophisticated logic reflected in the following script:

#!/bin/bash
set -euo pipefail

# Function to clean up tarball and extracted package
cleanup() {
rm *.tgz
rm -fR package
}

# Function to calculate the checksum of a package tarball
calculate_checksum() {
local tarball=$(ls *.tgz | head -n 1)
tar -xzf "$tarball"
cd package || exit 1
local checksum=$(\
tar \
--exclude='$lib' \
--sort=name \
--mtime='UTC 1970-01-01' \
--owner=0 \
--group=0 \
--numeric-owner -cf - . | sha256sum | awk '{print $1}')
cd ..
cleanup
echo "$checksum"
}

get_version() {
PACKAGE_VERSION=$(jq -r '.version' package.json)
}

publish() {
echo "Publishing new version: $PACKAGE_VERSION"
npm publish --access public --tag latest *.tgz
cleanup
exit 0
}

# Step 1: Read the package version from package.json
get_version
PACKAGE_NAME=$(jq -r '.name' package.json)

# Step 2: Check the latest version in the npm registry
LATEST_VERSION=$(npm show "$PACKAGE_NAME" version 2>/dev/null || echo "")

# Step 3: Prepare wing package
wing pack

# Step 4: If the versions are not equal, publish the new version
if [[ "$PACKAGE_VERSION" != "$LATEST_VERSION" ]]; then
publish
else
CURRENT_CHECKSUM=$(calculate_checksum)
# Download the latest package tarball
npm pack "$PACKAGE_NAME@$LATEST_VERSION" > /dev/null 2>&1
LATEST_CHECKSUM=$(calculate_checksum)
# Step 5: Compare the checksums
if [[ "$CURRENT_CHECKSUM" == "$LATEST_CHECKSUM" ]]; then
echo "No changes detected. Checksum matches the latest published version."
exit 0
else
echo $CURRENT_CHECKSUM
echo $LATEST_CHECKSUM
echo "Checksums do not match. Bumping patch version..."
npm version patch
wing pack
get_version
publish
fi
fi

Here is a brief explanation of what happens in this script:

  1. Step 1: Using the [jq](https://jqlang.github.io/jq/) command, extract the package name and version from the package.json file.
  2. Step 2: Using the [npm show](https://docs.npmjs.com/cli/v10/commands/npm-view) command, extract the package version number from the registry.
  3. Step 3: Using the [wing pack](https://www.winglang.io/docs/libraries) command, prepare the package .tgz file.
  4. Step 4: If version numbers differ, publish the new version using the [npm publish](https://docs.npmjs.com/cli/v10/commands/npm-publish) command.
  5. Step 5: If the versions are equal, calculate the checksum for the current and most recently published package. If the checksum values are equal, do nothing. Otherwise, using the [npm version patch](https://docs.npmjs.com/cli/v10/commands/npm-version) command, automatically bump up the [patch](https://symver.org/) version number, rebuild the .tgz file, and publish the new version.

Reliable checksum validation was the most challenging part of developing this script. The wing pack command creates a special @lib folder within the resulting .tgz archive. This folder introduces some randomness and can be affected by several factors, including Winglang compiler upgrades. Additionally, the .tgz file checksum calculation is sensitive to the order and timestamps of individual files. As a result, comparing the results of direct checksum calculation for the current and published packages was not an option.

To overcome these limitations, new archives are created with the @lib folder excluded and file order and timestamps normalized. The assistance of the ChatGPT 4o tool proved instrumental, especially in addressing this challenge.

In the current implementation, I keep this script in my home directory and invoke it from a common [Build.mk](http://Build.mk) Makefile used for all libraries (this might change in the future):

.PHONY: all compile-deps build-ts prepare test publish

all: publish

update-deps:
npm install && npm update

compile-ts: update-deps
ifneq ($(wildcard tsconfig.json),)
@echo "tsconfig.json found, running tsc..."
tsc
else
@echo "tsconfig.json not found, skipping TypeScript compilation."
endif

test: compile-ts
wing test -t sim ./test/*.test.w

publish: test
~/publish-npm.sh

Justification

To explain why I chose this particular way of publishing logic, I need to explain my overall project structure, illustrated in the diagram below:

The top of the diagram above reflects the NMP packages involved and their dependencies are depicted at the top, while the bottom part reflects my project folder structure.

The endor package is the ultimate goal of this development activity: an exploratory middleware framework for the Winglang programming language. Its efficacy is validated by a separate todo.endor.w application. Initially, both modules were kept together. However, keeping pure application parts separate from the infrastructure became progressively challenging.

The endor package uses three auxiliary packages logging , exception, and datetimex. These three packages are potential candidates to be contributed to the Winglibs project. However, they are still under active experimentation and development and are kept within the same Github repository.

Additionally, the endor package depends on other packages published on the public [npmjs](https://docs.npmjs.com/cli/v8/using-npm/registry) registry. Some of these packages, such as dynamodb and jwt belong to the same @winglibs namespace, while others do not.

I face a mixed-case challenge: the system already has a modular structure, but all components are under intensive development, requiring instant propagation of changes. As a solo developer and researcher, I still do not need more sophisticated CI/CD solutions, but rather employ a master Makefile to pull everything together:

.PHONY: all \
update_npm \
update_wing \
update_tsc \
make_datetimex \
make_exception \
make_logging \
make_endor

all: update_wing make_endor

update_npm:
sudo npm update -g npm

update_tsc: update_npm
sudo npm update -g tsc

update_wing: update_tsc
sudo npm update -g winglang

make_datetimex:
$(MAKE) -C ./datetimex -f ../Build.mk

make_logging: make_datetimex
$(MAKE) -C ./logging -f ../Build.mk

make_exception:
$(MAKE) -C ./exception -f ../Build.mk

make_endor: make_exception make_logging
$(MAKE) -C ./endor -f ../Build.mk

The todo.endor.w Makefile looks like this:

.PHONY: all update_wing install_endor test_local 

cloud ?= aws
target := target/main.tf$(cloud)

update_npm:
sudo npm update -g npm

update_wing: update_npm
sudo npm update -g winglang

install_endor:
npm install && npm update

build_ts:
tsc

test_local: update_wing install_endor build_ts test_app

test_app:
wing test -t sim ./test/*.w

test_remote:
wing test -t tf-$(cloud) ./test/service.test.w

run_local:
wing run -t sim ./dev.main.w

compile:
wing compile ./main.w -t tf-$(cloud)

tf-init: compile
( \
cd $(target) ;\
terraform init \
)

deploy: tf-init
( \
cd $(target) ;\
terraform apply -auto-approve \
)

destroy:
( \
cd $(target) ;\
terraform destroy -auto-approve \
)

This arrangement allows me to keep modules isolated, make changes in several places where appropriate, and perform fully automated build and verification without needing manual version updates within multiple package.json files.

Specifying cross-package dependencies is another point to pay attention to. Here is the endor package specification:

{
"name": "@winglibs/endor",
"description": "Wing middleware framework library",
"repository": {
"type": "git",
"url": "https://github.com/asterkin/endor.w.git",
"directory": "endor"
},
"version": "0.0.19",
"author": {
"email": "asher.sterkin@gmail.com",
"name": "Asher Sterkin"
},
"license": "MIT",
"peerDependencies": {
"@authenio/samlify-node-xmllint": "2.x.x",
"@winglibs/dynamodb": "0.x.x",
"@winglibs/jwt": "0.x.x",
"qs": "6.x.x",
"samlify": "2.x.x",
"ws": "8.x.x",
"inflection": "3.x.x",
"@winglibs/exception": "0.x.x",
"@winglibs/logging": "0.x.x"
}
}

Notice that, unlike traditional formats, all dependencies are specified using the x placeholder without the leading ^ symbol. This is because, with the ^ prefix included, the most up-to-date versions are brought in only for the final todo.endor.w application, whereas I needed them to be used in the dependent modules' unit tests. Using the x placeholder instead does the job.

In summary, while not final, the described solution provides good enough treatment for all essential requirements at the current stage of the system evolution. As the system grows, adequate adjustments will be implemented and reported. Stay tuned.

Acknowledgments

Throughout the preparation of this publication, I utilized several key tools to enhance the draft and ensure its quality.

The initial draft was crafted with the organizational capabilities of Notion's free subscription, facilitating the structuring and development of ideas.

For grammar and spelling review, the free version of Grammarly proved useful for identifying and correcting basic errors, ensuring the readability of the text.

The enhancement of stylistic expression and the narrative coherence checks were performed using the paid version of ChatGPT 4o. The ChatGPT 4o tool was also used for developing the package publishing script and creation of NMP elements icons.

While these advanced tools and resources significantly contributed to the preparation process, the concepts, solutions, and final decisions presented in this article are entirely my own, for which I bear full responsibility.

In Search for Winglang Middleware

· 24 min read
Asher Sterkin
Technical Writer
Part Two: Pipeline Formation with Template Method

Asher&#39;s blog cover art

Winglang's unique capability to uniformly handle both preflight (cloud resource configuration) and inflight (cloud events processing) logic opens up meta-programming possibilities akin to Lisp macros. This allows for the dynamic adjustment of service configurations to various deployment targets—such as DEV, TEST, STAGE, and PROD—at the build stage. By doing so, it optimizes cost, security, and performance without compromising the integrity of the core service logic, which remains largely insulated from middleware framework details. This level of flexibility is unmatched by more traditional cloud middleware libraries, such as PowerTools for AWS Lambda, which I explored in the first part of this series.

In this part, I will explore how a middleware framework can leverage the Template Method Design Pattern. This design pattern has proven instrumental in defining the common elements of the REST API Create/Retrieve/Update/Delete (CRUD) request handling flow, while still allowing enough flexibility to accommodate the specifics of each request.

Specifically, the application of the Template Method Design Pattern to define a common request-handling workflow has demonstrated the following benefits:

  • Unlike Decorator, it organically presents post-processing steps of request handling in their natural sequence.
  • It facilitates the reuse of a common request-handling definition across all functions related to a resource, service, or even across all services developed by the same team or organization.
  • It enables the specification of multiple configurations optimized for various deployment targets (DEV, TEST, STAGE, PROD), addressing variability at the build stage to eliminate unnecessary overhead and security risks.

Nevertheless, this approach has limitations. A high number of deployment targets, services, resources, and function permutations may necessitate maintaining a large number of templates—a common challenge in any Engineering Platform based on blueprints.

Exploring whether these limitations can be surmounted using Winglang's unique capabilities will be the focus of future research.

As clarified in the previous publication,

Common middleware services augment distribution middleware by defining higher-level domain-independent reusable services that allow application developers to concentrate on programming business logic, without the need to write the “plumbing” code required to develop distributed applications via lower-level middleware directly.

In our pursuit, we specifically aim to define common service configurations that are adaptable to various development targets. Such configurations are not only reusable across multiple services and resources within the same team or organization but are also distinct enough not to be overgeneralized in the form of a framework but rather to merit integration into a tailored Engineering Platform. This balance ensures that while the configurations maintain a high level of generality, they remain sufficiently detailed to support the unique needs of different projects within the organization.

Here is an example of how common service configurations can be implemented using Winglang. Typically, this or a similar code snippet will be a part of a larger solution integrated into an organization's or team's engineering platform:

bring endor;
bring cloud;
bring logging;

//Could be a part of an organization or team engineering platform
pub class ServiceFactory impl endor.IRestApiHandlerTemplate {
_tools: endor.ApiTools;
_mode: endor.Mode;
pub logger: logging.Logger;
pub api: cloud.Api;

_getLoggingLevel(mode: endor.Mode): logging.Level {
if mode == endor.Mode.PROD {
return logging.Level.INFO;
} elif mode == endor.Mode.DEV {
return logging.Level.TRACE;
} else {
return logging.Level.DEBUG;
}
}

_getToolsOptions(
mode: endor.Mode,
logger: logging.Logger
): endor.ApiToolsOptions {
if mode == endor.Mode.DEV {
return endor.ApiToolsOptions{
logger: logger,
statusMessage: Map<str>{} // leave original error messages intact
};
}
return endor.ApiToolsOptions{
logger: logger
}; // use defaults
}

new(serviceName: str, mode: endor.Mode) {
this._mode = mode;
this.logger = new logging.Logger(
this._getLoggingLevel(mode),
serviceName);
this._tools = new endor.ApiTools(
this._getToolsOptions(mode,
this.logger));
this.api = new cloud.Api();
}

_getProdApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?
): endor.RestApiBuilder {
let tokenFactory = new endor.FixedSecretTokenFiltersFactory();
let cookieFactory = new endor.CookieAuthFiltersFactory(tokenFactory);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
cookieFactory,
responseFormats);
if getHomePage != nil {
apiBuilder.samlLogin(cookieFactory, getHomePage!);
}
return apiBuilder;
}
_getDevApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?
): endor.RestApiBuilder {
let session = Json {
userID: "test-user",
fullName: "Test User"
};
let stubAuth = new endor.ApiStubAuthFactory(session);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
stubAuth,
responseFormats);
if getHomePage != nil {
apiBuilder.stubLogin(session, getHomePage!);
}
return apiBuilder;
}
_getDefaultApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
password: str
): endor.RestApiBuilder {
let basicAuth = new endor.ApiBasicAuthFactory(password);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
basicAuth,
responseFormats);
return apiBuilder;
}
pub getApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?,
password: str?
): endor.RestApiBuilder {
if this._mode == endor.Mode.PROD {
return this._getProdApiBuilder(
resource,
responseFormats,
getHomePage);
} elif this._mode == endor.Mode.DEV {
return this._getDevApiBuilder(
resource,
responseFormats,
getHomePage);
} else {
return this._getDefaultApiBuilder(
resource,
responseFormats,
password!);
}
}

pub makeRequestHandler(
functionName: str,
proc: inflight (cloud.ApiRequest): cloud.ApiResponse
): inflight(cloud.ApiRequest): cloud.ApiResponse {
let handler = inflight (request: cloud.ApiRequest): cloud.ApiResponse => {
let var response = cloud.ApiResponse{};
try {
this._tools.logRequest(functionName, request);
response = proc(request);
} catch err {
response = this._tools.errorResponse(err);
}
this._tools.logResponse(functionName, response);
response = this._tools.responseMessage(response);
return response;
};
return handler;
}

}

This example is quite detailed, so we will break it down section by section to fully understand its structure and functionality.

This module, by convention named middleware.w, features a Winglang preflight class called ServiceFactory. This class encapsulates the following public resources and methods, which are crucial for the middleware's operation:

makeRequestHandler() Factory Method

    pub makeRequestHandler(
functionName: str,
proc: inflight (cloud.ApiRequest): cloud.ApiResponse
): inflight(cloud.ApiRequest): cloud.ApiResponse {
let handler = inflight (request: cloud.ApiRequest): cloud.ApiResponse => {
let var response = cloud.ApiResponse{};
try {
this._tools.logRequest(functionName, request);
response = proc(request);
} catch err {
response = this._tools.errorResponse(err);
}
this._tools.logResponse(functionName, response);
response = this._tools.responseMessage(response);
return response;
};
return handler;
}

This Factory Method plays a central role in implementing the Common Service Middleware. It is a Winglang preflight method that obtains two parameters:

  • functionName: The name of the function handling the API request, used for logging purposes.
  • proc: a Winglang inflight function that transforms cloud.ApiRequest into a cloud.ApiResponse

Internally, it defines a Winglang inflight function, called handler that encapsulates a typical API request processing workflow:

  1. Logging the Request: Initially logs the incoming cloud.ApiRequest.
  2. Processing the Request: Executes the proc function to obtain a cloud.ApiResponse.
  3. Error Handling: In the case of an error, it transforms the error into an appropriate cloud.ApiResponse.
  4. Response Logging: Logs the outgoing cloud.ApiResponse.
  5. Error Message Management: Modifies the error message in the response based on the configuration to prevent leakage of sensitive information to potential attackers.

Operations for logging and error handling are managed using the auxiliary endor.ApiTools class, which we will explore in further detail later.

getApiBuilder() Factory Method

The getApiBuilder() method, another key component implemented as a Factory Method, dynamically creates a properly configured API Builder for a specific resource, depending on the deployment mode:

pub getApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?,
password: str?
): endor.RestApiBuilder {
if this._mode == endor.Mode.PROD {
return this._getProdApiBuilder(
resource,
responseFormats,
getHomePage);
} elif this._mode == endor.Mode.DEV {
return this._getDevApiBuilder(
resource,
responseFormats,
getHomePage);
} else {
return this._getDefaultApiBuilder(
resource,
responseFormats,
password!);
}
}

This Winglang preflight method accepts four parameters:

  • resource: A Winglang Struct defining the API resource, including its names and HTTP paths for singular and plural operations.
  • responseFormats: A Winglang Array specifying supported response formats, such as application/json, text/html, or text/plain.
  • getHomePage: An optional Winglang inflight function to fetch the home page content using session data and the required format.
  • password: An optional string for temporary use in HTTP Basic Authentication.

Based on the _mode field, the method directs the construction of the appropriate builder:

  • It calls _getProdApiBuilder for the PROD mode.
  • It invokes _getDevApiBuilder for the DEV mode.
  • It defaults to _getDefaultApiBuilder in other cases.

The main variability point that distinguishes these three options is the authentication strategy applied to each API request. Let's examine the specifics of each builder's implementation.

_getProdApiBuilder() Factory Method

_getProdApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?
): endor.RestApiBuilder {
let tokenFactory = new endor.FixedSecretTokenFiltersFactory();
let cookieFactory = new endor.CookieAuthFiltersFactory(tokenFactory);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
cookieFactory,
responseFormats);
if getHomePage != nil {
apiBuilder.samlLogin(cookieFactory, getHomePage!);
}
return apiBuilder;
}

For production environments, the security of API requests is paramount. This method implements stringent authentication protocols using JSON Web Tokens (JWT) embedded within Cookie HTTP Headers. The JWTs are signed with a fixed random key, a cost-effective measure that maintains robust security for services at this level.

This Winglang preflight method sets up the described security configuration. Additionally, if the getHomePage parameter is not nil, the method configures an extra HTTP request handler for SAML-based authentication, offering another layer of security and user verification (further details on this process can be found here).

_getDevApiBuilder() Factory Method

_getDevApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
getHomePage: (inflight (Json, str): str)?
): endor.RestApiBuilder {
let session = Json {
userID: "test-user",
fullName: "Test User"
};
let stubAuth = new endor.ApiStubAuthFactory(session);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
stubAuth,
responseFormats);
if getHomePage != nil {
apiBuilder.stubLogin(session, getHomePage!);
}
return apiBuilder;
}

For development purposes, especially when using the Winglang Simulator in interactive mode, security requirements can be relaxed. In this mode, authentic security is not required, allowing developers to focus on functionality and flow without the overhead of complex security protocols.

This Winglang preflight method implements such a setup. It uses a stub authentication system based on predefined user session data. Furthermore, if the getHomePage parameter is supplied and not nil, the method adds an HTTP request handler that simulates the login process, maintaining the integrity of the user experience even in a simulated environment.

_getDefaultApiBuilder() Factory Method

_getDefaultApiBuilder(
resource: endor.ApiResource,
responseFormats: Array<str>,
password: str
): endor.RestApiBuilder {
let basicAuth = new endor.ApiBasicAuthFactory(password);
let apiBuilder = new endor.RestApiBuilder(
this.api,
resource,
this,
basicAuth,
responseFormats);
return apiBuilder;
}

For test and stage modes, where end-to-end testing is routinely performed, a balance between security, cost-efficiency, and performance is essential. Unlike local simulations that utilize stub authentication similar to the development environment, end-to-end tests in real cloud platforms like AWS require more robust security.

This Winglang preflight method achieves this by implementing HTTP Basic Authentication. It utilizes a dynamically generated password to ensure security while maintaining cost efficiency. The use of HTTP Basic Authentication, where credentials are passed directly in the header, eliminates the need for separate login request handling, streamlining the testing process.

ServiceFactory Object Initialization

Proper initialization of the ServiceFactory object is crucial for the functionality of its public methods and fields. Below is a detailed look into its initialization process:

bring endor;
bring cloud;
bring logging;

//Could be a part of organization or team engineering platform
pub class ServiceFactory impl endor.IRestApiHandlerTemplate {
_tools: endor.ApiTools;
_mode: endor.Mode;
pub logger: logging.Logger;
pub api: cloud.Api;

_getLoggingLevel(mode: endor.Mode): logging.Level {
if mode == endor.Mode.PROD {
return logging.Level.INFO;
} elif mode == endor.Mode.DEV {
return logging.Level.TRACE;
} else {
return logging.Level.DEBUG;
}
}

_getToolsOptions(
mode: endor.Mode,
logger: logging.Logger
): endor.ApiToolsOptions {
if mode == endor.Mode.DEV {
return endor.ApiToolsOptions{
logger: logger,
statusMessage: Map<str>{} // leave original error messages intact
};
}
return endor.ApiToolsOptions{
logger: logger
}; // use defaults
}

new(serviceName: str, mode: endor.Mode) {
this._mode = mode;
this.logger = new logging.Logger(
this._getLoggingLevel(mode),
serviceName);
this._tools = new endor.ApiTools(
this._getToolsOptions(mode,
this.logger));
this.api = new cloud.Api();
}

The new() constructor method is designed to take two parameters:

  • serviceName: Utilized in all log messages to identify service-specific operations.
  • mode: Determines the appropriate configuration settings for different operational environments.

The initialization sequence performs the following steps:

  1. Mode Configuration: Stores the mode to dictate the behavior of the getApiBuilder() method.
  2. Logger Initialization: Creates a Logger object with a logging level based on mode:
    • INFO for PROD for streamlined logging.
    • TRACE for DEV to enable detailed debugging.
    • DEBUG for other environments to balance detail and performance.
  3. ApiTools Configuration: Establishes an endor.ApiTools object with settings influenced by mode:
    • Retains original error messages in DEV mode to aid in debugging.
    • Replaces error messages with standard HTTP response text in other modes to safeguard against potential security risks.

This ServiceFactory class, by implementing the endor.IRestApiHandlerTemplate interface, seamlessly integrates with the endor.RestApiBuilder. This allows the latter to utilize the makeRequestHandler() method of ServiceFactory, thus ensuring consistent handling of all API requests.

TodoService

The common middleware service configuration described previously hides a substantial portion of the system infrastructure complexity, allowing specific service configurations to be defined with ease and precision. A practical implementation of this approach can be seen in the TodoService, first introduced in the previous publication:

bring endor;
bring cloud;
bring logging;
bring "./core" as core;
bring "./adapters" as adapters;
bring "./middleware.w" as middleware;

pub class TodoService {
_api: cloud.Api;

//TODO: true content negotiation; unit test?; move to adapters??
_getResponseFormatters(
mode: endor.Mode,
resource: endor.ApiResource
): Map<core.ITodoFormatter> {
if mode == endor.Mode.TEST {
return Map<core.ITodoFormatter> {
"application/json" => new adapters.TodoJsonFormatter()
};
}
return Map<core.ITodoFormatter> {
"text/html" => new adapters.TodoHtmlFormatter(resource.htmlPath),
"text/plain" => new adapters.TodoTextFormatter(),
};
}

new(mode: endor.Mode, password: str?) {
let serviceName = "Todo Service";
let factory = new middleware.ServiceFactory(serviceName, mode);
let resource = new endor.ApiResource("Task");
let responseFormatters = this._getResponseFormatters(mode, resource);
let repository = new adapters.TaskTableRepository(factory.logger);
let handler = new core.TodoHandler(
repository,
responseFormatters,
factory.logger
);
let apiBuilder = factory.getApiBuilder(
mode,
resource,
responseFormatters.keys(),
handler.getHomePage(),
password
);

apiBuilder.retrieveResources(handler.getAllTasks());
apiBuilder.createResource(handler.createTask());
apiBuilder.replaceResource(handler.replaceTask());
apiBuilder.deleteResource(handler.deleteTask());
this._api = factory.api;
}

pub getUrl(): str {
return this._api.url;
}
}

The TodoService class serves as a Winglang preflight entity that orchestrates the integration of the service core, its adapters, and middleware. It includes a public getUrl() method for testing purposes.

Service Initialization

The new() constructor method takes two parameters:

  • mode: Determines the output format selection and middleware configuration.
  • password: An optional string for HTTP Basic Authentication in scenarios requiring secure access.

The service is initialized through the following steps:

  1. Middleware Configuration: Initializes a ServiceFactory with serviceName and mode.
  2. Resource Descriptor: Sets up a Task resource descriptor that translates to the /tasks HTTP path.
  3. Response Formatter Configuration: Depending on the mode, it configures response formatters:
    • application/json for TEST mode.
    • text/html and text/plain for PROD and DEV modes.
  4. Repository Creation: Establishes a Tasks repository using a DynamoDB backend.
  5. Handler Setup: Configures a core.TodoHandler with necessary dependencies, including a logger from ServiceFactory.
  6. API Builder Setup: Uses ServiceFactory.getApiBuilder() to link TodoHandler methods to respective REST API endpoints.
  7. REST API Wiring: Connects TodoHandler methods to the corresponding REST API calls.
  8. API Object Storage: Retains the cloud.Api object to handle URL retrieval via getUrl().

Configurations

This TodoService can be instantiated with different target modes and optionally a random password, resulting in three primary configurations for various environments:

  1. Production: let _service = new service.TodoService(endor.Mode.PROD);
  2. Development: let _service = new service.TodoService(endor.Mode.DEV);
  3. Testing: let _service = new service.TodoService(endor.Mode.TEST, _password);

Design Diagram

While detailed code snippets and textual descriptions provide precise insight into design decisions, they often fall short of conveying a holistic view. For a broader perspective, visual representations are invaluable, especially when dealing with concepts as abstract as higher-order programming utilized by Winglang's preflight/inflight mechanisms. This section aims to bridge this understanding gap through graphical illustrations.

To visualize core system components and their relationships a UML Class diagram is a suitable tool:

Design Pattern 1

In this diagram, solid arrows indicate permanent references, dashed arrows depict temporal interactions, and diamond-ended lines show component aggregations. This static representation helps delineate how components are interconnected during the preflight phase.

However, to visualize what happens dynamically at the inflight stage, a more specialized notation is necessary. Traditional UML communication and sequence diagrams were found inadequate, prompting the creation of a custom notation specifically designed for this purpose.

Illustrated below is the createTask with HTTP Basic Authentication scenario:

Design Pattern 2 In this diagram:

  • Circles with transparent backgrounds represent individual functional steps.
  • Circles with grey backgrounds denote template method plug-in function sockets.
  • Rounded rectangles indicate top-level functions.
  • Dashed arrows symbolize standard function calls, while solid arrows indicate indirect function calls via function references.

Interpretation of the Diagram

  1. An incoming HTTP POST request is received by the cloud resource (e.g., AWS API Gateway), which uses the Winglang cloud.Api to convert it into a cloud.ApiRequest and invokes the appropriate inflight function, typically specified within the RestApiHandlerTemplate (here, implemented by ServiceFactory).
  2. The api request handling process begins within a try block, where the logRequest function from ApiTools logs the request, and the plugged-in function (createResource from RestApiBuilder) processes the request. Upon error, errorResponse from ApiTools converts exceptions into cloud.ApiResponse. Regardless of the outcome, logResponse and responseMessage from ApiTools are invoked to finalize the processing.
  3. The createResource function within RestApiBuilder initiates authentication via a plugged-in auth function (provided by BasicAuthFactory), which extracts user data and passes control to createResource of RestApiFactory.
  4. Finally, createResource in RestApiFactory parses HTTP headers and body data and calls the createTask function provided by TodoHandler, which handles the core business logic.

This dynamic interaction is indeed complex, highlighting the intricate and interconnected nature of the system. The proposed middleware design aims to shield users from this complexity in daily operations. However, when issues arise, creating a precise graphical representation of the underlying system dynamics becomes essential. Future research will likely focus on developing tools to maintain cognitive control over such complex systems, ensuring that developers can effectively manage and troubleshoot without being overwhelmed.

Todo Service Core

The architecture adopted allows the service core to remain independent of any middleware framework, focusing on core functionality without being tangled in middleware specifics. An example of this is the core.TodoHandler class:

bring logging;
bring "./task.w" as task;
bring "./parser.w" as parser;
bring "./formatter.w" as formatter;

// Experimental implementation of
// "Preflight Object Oriented, Inflight Functional"
// Design Pattern
pub class TodoHandler {
_tasks: task.ITaskDataRepository;
_parser: parser.TodoParser;
_formatter: formatter.TodoFormattingRouter;
_logger: logging.Logger;

new(
tasks_: task.ITaskDataRepository,
formatters: Map<formatter.ITodoFormatter>,
logger: logging.Logger
) {
this._tasks = tasks_;
this._parser = new parser.TodoParser();
this._formatter = new formatter.TodoFormattingRouter(formatters);
this._logger = logger;
}

pub getHomePage(): inflight (Json, str): str {
let handler = inflight (user: Json, outFormat: str): str => {
let userData = this._parser.parseUserData(user);

return this._formatter.formatHomePage(outFormat, userData);
};
return handler;
}

pub getAllTasks(): inflight (Json, Map<str>, str): str {
let handler = inflight (
user: Json,
query: Map<str>,
outFormat: str
): str => {
let userData = this._parser.parseUserData(user);
//TBD: should it get userData instead?
let tasks = this._tasks.getTasks(userData.userID);

return this._formatter.formatTasks(outFormat, tasks);
};
return handler;
}

pub createTask(): inflight (Json, Json, str): str {
let handler = inflight (
user: Json,
taskAttributes: Json,
outFormat: str
): str => {
let taskData = this._parser.parsePartialTaskData(
user,
taskAttributes);
this._tasks.addTask(taskData);
//TBD: cloud events?
this._logger.info(
"createTask",
Json{userID: taskData.userID, taskID:taskData.taskID});

return this._formatter.formatTasks(outFormat, [taskData]);
};
return handler;
}

pub replaceTask(): inflight (Json, str, Json, str): str {
let handler = inflight (
user: Json,
id: str,
taskAttributes: Json,
outFormat: str
): str => {
let taskData = this._parser.parseFullTaskData(
user,
id,
taskAttributes);
this._tasks.replaceTask(taskData);
//TBD: cloud events?
this._logger.info(
"replaceTask",
Json{userID: taskData.userID, taskID:taskData.taskID});

return this._formatter.formatTasks(outFormat, [taskData]);
};
return handler;
}

pub deleteTask(): inflight (Json, str): str {
let handler = inflight (user: Json, id: str): str => {
let userData = this._parser.parseUserData(user);
let taskID = num.fromStr(id);
//TBD: taskKey? userData?
this._tasks.deleteTask(userData.userID, taskID);
//TBD: cloud events?
this._logger.info(
"deleteTask",
Json{userID: userData.userID, taskID:taskID});

return ""; //TBD: formatter?
};
return handler;
}
}

The core.TodoHandler is designed as a Winglang preflight class that encapsulates key functionalities for Todo Service operations. Each method in this class exemplifies a Factory Method, returning specialized inflight functions that handle specific aspects of Todo management.

The only implicit coupling between the core of the Todo Service and its middleware lies in the parameters passed to each function. This level of coupling, referred to as Knowledge Sharing, is a trade-off typically considered acceptable in such architectural designs, facilitating seamless integration while maintaining a clear separation of concerns.

Interestingly enough, introducing Generics support in Winglang could potentially increase rather than decrease system coupling. With Generics, the coordination between the core and middleware layers would extend beyond just the order and types of parameters. It would also necessitate sharing the names of functions and their parameters, thereby tightening the interdependence within the system.

The following UML class diagram summarizes the Todo Service logic design in visual form:

ToDo Service

A detailed description of this design is presented in a previous publication.

Endor Middleware Framework

The ServiceFactory class, detailed earlier, relies on the Endor middleware framework—an experimental library designed to push the boundaries of what is possible with Winglang, a new cloud-oriented programming language.

The name "Endor", derived from Quenya—a functional language created by J.R.R. Tolkien for the Elves in his Middle-earth fiction—translates to "Middle-earth." This nomenclature not only signifies 'middle' but also metaphorically represents our exploration into Winglang's unleashed yet potential, positioning it as a pioneering language at the crossroads of established practices and innovative paradigms.

In its current iteration, the Endor middleware framework encapsulates an initial set of functionalities for HTTP request handling, including various authentication methods. While a comprehensive review of the entire framework is outside the scope of this publication, we will briefly explore the Endor.ApiBuilder class implementation, which plays a crucial role in integrating application-specific handlers into a common request processing infrastructure:

bring cloud;
bring "./apiStubAuth.w" as apiStubAuth;
bring "./apiResource.w" as apiResource;
bring "./apiAuthFactory.w" as authFactory;
bring "./restApiFactory.w" as restApiFactory;
bring "./cookieAuthFilters.w" as cookieFilters;

pub interface IRestApiHandlerTemplate {
makeRequestHandler(
functionName: str,
proc: inflight (cloud.ApiRequest): cloud.ApiResponse
): inflight (cloud.ApiRequest): cloud.ApiResponse;
}

pub class RestApiBuilder {
_resource: apiResource.ApiResource;
_template: IRestApiHandlerTemplate;
_factory: restApiFactory.RestApiFactory;
_auth: (inflight (cloud.ApiRequest): Json);
_api: cloud.Api;

new(
api: cloud.Api,
resource: apiResource.ApiResource,
template: IRestApiHandlerTemplate,
authFactory: authFactory.IApiAuthFactory,
responseFormats: Array<str>
) {
this._resource = resource;
this._template = template;
this._factory = new restApiFactory.RestApiFactory(responseFormats);
this._auth = authFactory.getAuth();
this._api = api;
}

_makeAuthRequestHandler(
functionName: str,
proc: inflight (Json, cloud.ApiRequest): cloud.ApiResponse
): inflight(cloud.ApiRequest): cloud.ApiResponse {
let handler = inflight(request: cloud.ApiRequest): cloud.ApiResponse => {
let userData = this._auth(request);
return proc(userData, request);
};
return this._template.makeRequestHandler(functionName, handler);
}

pub samlLogin(
cookieFactory: cookieFilters.CookieAuthFiltersFactory,
getHomePage: inflight (Json, str): str
): void {
this._api.post(
"/sp/acs",
this._template.makeRequestHandler(
"login",
this._factory.samlLogin(cookieFactory, getHomePage)
)
);
}

pub stubLogin(
session: Json,
getHomePage: inflight (Json, str): str
): void {
this._api.get(
"/",
this._template.makeRequestHandler(
"login",
this._factory.stubLogin(session, getHomePage)
)
);
}

pub retrieveResources(
handler: inflight (Json, Map<str>, str): str
): void {
this._api.get(
this._resource.path,
this._makeAuthRequestHandler(
"get{this._resource.plural}",
this._factory.retrieveResources(
handler
)
)
);
}

pub createResource(
handler: inflight (Json, Json, str): str
): void {
this._api.post(
this._resource.path,
this._makeAuthRequestHandler(
"create{this._resource.singular}",
this._factory.createResource(
handler
)
)
);
}

pub replaceResource(
handler: inflight (Json, str, Json, str): str
): void {
this._api.put(
this._resource.idPath,
this._makeAuthRequestHandler(
"replace{this._resource.singular}",
this._factory.replaceResource(
handler
)
)
);
}

pub deleteResource(
handler: inflight (Json, str): str
): void {
this._api.delete(
this._resource.idPath,
this._makeAuthRequestHandler(
"delete{this._resource.singular}",
this._factory.deleteResource(
handler
)
)
);
}

//TODO: other operations: partial update (patch), has (head), delete all, update all
}

The RestApiBuilder class is designed to wrap application-specific handlers, such as createTask(), within a standardized cloud API request-response conversion process, seamlessly incorporating specified authentication methods. The conversion from cloud.ApiRequest to cloud.ApiResponse is further handled by the endor.RestApiFactory, whose details are not covered in this publication.

The endor.ApiTools class, another significant component of the framework, provides a suite of middleware operations:

bring cloud;
bring logging;
bring exception;

pub struct ApiToolsOptions {
logger: logging.Logger;
logLevel: logging.Level?;
statusMessage: Map<str>?;
statusLogging: Map<logging.Level>?;
}

//TODO: unit test
pub class ApiTools {
_logger: logging.Logger;
_logLevel: logging.Level;
_errorStatus: Map<num>;
_statusMessage: Map<str>;
_statusLogging: Map<logging.Level>;

new(
options: ApiToolsOptions
) {
this._logger = options.logger;
this._logLevel = options.logLevel ?? logging.Level.DEBUG;
this._errorStatus = Map<num>{
"ValueError" => 400,
"AuthenticationError" => 401,
"AuthorizationError" => 403,
"KeyError" => 404,
"InternalError" => 500,
"NotImplementedError" => 501
};
this._statusMessage = options.statusMessage ?? Map<str> {
"400" => "Bad Request",
"401" => "Unauthorized",
"403" => "Forbidden",
"404" => "Not Found",
"500" => "Internal Server Error",
"501" => "Not Implemented"
};
this._statusLogging = options.statusLogging ?? Map<logging.Level> {
"200" => logging.Level.DEBUG,
"400" => logging.Level.WARNING,
"401" => logging.Level.WARNING,
"403" => logging.Level.WARNING,
"404" => logging.Level.WARNING,
"500" => logging.Level.ERROR,
"501" => logging.Level.ERROR,
};
}
pub inflight logRequest(functionName: str, request: cloud.ApiRequest): void {
this._logger.log(
this._logLevel,
functionName,
Json{
request: Json.parse(Json.stringify(request))
}
);
}
pub inflight logResponse(functionName: str, response: cloud.ApiResponse): void {
if let logLevel = this._statusLogging.tryGet("{response.status!}") {
this._logger.log(
logLevel,
functionName,
Json{
response: Json.parse(Json.stringify(response))
}
);
}
}
pub inflight errorResponse(err: str): cloud.ApiResponse {
if let error = exception.tryParse(err) {
if let status = this._errorStatus.tryGet(error.tag) {
return cloud.ApiResponse {
status: status,
headers: {
"Content-Type" => "text/plain"
},
body: error.message
};
}
}
return cloud.ApiResponse {
status: 500,
headers: {
"Content-Type" => "text/plain"
},
body: err
};
}
pub inflight responseMessage(response: cloud.ApiResponse): cloud.ApiResponse {
if let message = this._statusMessage.tryGet("{response.status!}") {
return cloud.ApiResponse{
status: response.status,
headers: response.headers,
body: message
};
}
return response;
}
}

This class offers essential functionalities for logging requests and responses, handling errors, and modifying response messages based on the status codes, thereby standardizing the error handling and logging processes across all middleware services.

The following UML class diagram summarizes the endor middleware framework design in visual form:

Service Factory

Middleware Service Layers

Finally, let us synthesize the components discussed throughout this series and identify some common patterns that have emerged. Below, we visualize how these elements interact:

Middleware Layer Revised

At the top of this structure is located the TodoService which pulls together three critical elements:

  1. The Service Core - Defines the fundamental service logic independent of specific communication and storage implementations:
    • Domain Entities like Task and User.
    • Domain Entity Factory - Constructs domain-specific data, such as assigning unique IDs to tasks.
    • Parser - Transforms JSON into domain-specific data structures.
    • Formatter - An abstract interface for converting domain data into required formats (e.g., HTML).
    • Repository - An abstract interface for domain data storage.
    • Handler - Integrates core components to manage specific API requests.
  2. The Service Adapters - Implements specific interfaces for the Formatter and Repository. Examples include:
    • HTML Formatter
    • JSON Formatter
    • Plain Text Formatter
    • Task Repository using DynamoDB.
  3. The Platform Middleware - Contains the ServiceFactory class, crucial for implementing a standardized request-handling workflow.

The endor Middleware Framework provides foundational building blocks for composing request handling workflows, utilizing core Winglang libraries like logging and exception for enhanced functionality.

Conclusions

The application of the Template Method Design Pattern to define a common request-handling workflow has demonstrated significant flexibility and utility, surpassing mainstream solutions such as PowerTools for AWS Lambda:

  • It organically presents post-processing steps of request handling in their natural sequence.
  • It facilitates the reuse of a common request-handling definition across all functions related to a resource, service, or even across all services developed by the same team or organization.
  • It enables the specification of multiple configurations optimized for various deployment targets (DEV, TEST, STAGE, PROD), addressing variability at the build stage to eliminate unnecessary overhead and security risks.

Nevertheless, this approach has limitations. A high number of deployment targets, services, resources, and function permutations may necessitate maintaining a large number of templates—a common challenge in any Engineering Platform based on blueprints.

Exploring whether these limitations can be surmounted using Winglang's unique capabilities will be the focus of future research. Stay tuned for further developments.

Acknowledgments

Throughout the preparation of this publication, I utilized several key tools to enhance the draft and ensure its quality.

The initial draft was crafted with the organizational capabilities of Notion's free subscription, facilitating the structuring and development of ideas.

For grammar and spelling review, the free version of Grammarly proved useful for identifying and correcting basic errors, ensuring the readability of the text.

The enhancement of stylistic expression and the narrative coherence checks were performed using the paid version of ChatGPT 4.0.

While these advanced tools and resources significantly contributed to the preparation process, the concepts, solutions, and final decisions presented in this article are entirely my own, for which I bear full responsibility.

In Search for Winglang Middleware Part One

· 23 min read
Asher Sterkin
Technical Writer

Introducing a new programming language that creates an opportunity and an obligation to reevaluate existing methodologies, solutions, and the entire ecosystem—from language syntax and toolchain to the standard library through the lens of first principles.

Simply lifting and shifting existing applications to the cloud has been broadly recognized as risky and sub-optimal. Such a transition tends to render applications less secure, inefficient, and costly without proper adaptation. This principle holds for programming languages and their ecosystems.

Currently, most cloud platform vendors accommodate mainstream programming languages like Python or TypeScript with minimal adjustments. While leveraging existing languages and their vast ecosystems has certain advantages—given it takes about a decade for a new programming language to gain significant traction—it's constrained by the limitations of third-party libraries and tools designed primarily for desktop or server environments, with perhaps a nod towards containerization.

Winglang is a new programming language pioneering a cloud-oriented paradigm that seeks to rethink the cloud software development stack from the ground up. My initial evaluations of Winglang's syntax, standard library, and toolchain were presented in two prior Medium publications:

  1. Hello, Winglang Hexagon!: Exploring Cloud Hexagonal Design with Winglang, TypeScript, and Ports & Adapters
  2. Implementing Production-grade CRUD REST API in Winglang: The First Steps

Capitalizing on this exploration, I will focus now on the higher-level infrastructure frameworks, often called 'Middleware'. Given its breadth and complexity, Middleware development cannot be comprehensively covered in a single publication. Thus, this publication is probably the beginning of a series where each part will be published as new materials are gathered, insights derived, or solutions uncovered.

Part One of the series, the current publication, will provide an overview of Middleware origins and discuss the current state of affairs, and possible directions for Winglang Middleware. The next publications will look at more specific aspects.

With Winglang being a rapidly evolving language, distinguishing the core language features from the third-party Middleware built atop this series will remain an unfolding narrative. Stay tuned.

Throughout the preparation of this publication, I utilized several key tools to enhance the draft and ensure its quality.

The initial draft was crafted with the organizational capabilities of Notion's free subscription, facilitating the structuring and development of ideas.

For grammar and spelling review, the free version of Grammarly proved useful for identifying and correcting basic errors, ensuring the readability of the text.

The enhancement of stylistic expression and the narrative coherence checks were performed using the paid version of ChatGPT 4.0.

I owe a special mention to Nick Gal’s informative blog post for illuminating the origins of the term "Middleware," helping to set the correct historical context of the whole discussion.

While these advanced tools and resources significantly contributed to the preparation process, the concepts, solutions, and final decisions presented in this article are entirely my own, for which I bear full responsibility.

What is Middleware?

The term "Middleware" passed a long way from its inception and formal definitions to its usage in day-to-day software development practices, particularly within web development.

Covering every nuance and variation of Middleware would be long a journey worthy of a comprehensive volume entitled “The History of Middleware”—a volume awaiting its author.

In this exploration, we aim to chart the principal course, distilling the essence of Middleware and its crucial role in filling the gap between basic-level infrastructure and the practical needs of cloud-based applications development.

Origins of Middleware

Brian RanellThe concept of Middleware ||traces its roots back to an intriguing figure: the Russian-born British cartographer and cryptographer, Alexander d’Agapeyeff, at the "1968 NATO Software Engineering Conference."

Despite the scarcity of official information about d’Agapeyeff, his legacy extends beyond the enigmatic d’Agapeyeff Cipher, as he also played a pivotal role in the software industry as the founder and chairman of the "CAP Group." Insights into the early days of Middleware are illuminated by Brian Randell, a distinguished British computer scientist, in his recounting of "Some Middleware Beginnings."

At the NATO Conference d’Agapeyeff introduced his Inverted Pyramid—a conceptual framework positioning Middleware as the critical layer bridging the gap between low-level infrastructure (such as Control Programs and Service Routines) and Application Programs:

Fig 1: Alexander d'Agapeyeff's Pyramid

Here is how A. d’Agapeyeff explains it:

An example of the kind of software system I am talking about is putting all the applications in a hospital on a computer, whereby you get a whole set of people to use the machine. This kind of system is very sensitive to weaknesses in the software, particular as regards the inability to maintain the system and to extend it freely.

This sensitivity of software can be understood if we liken it to what I will call the inverted pyramid... The buttresses are assemblers and compilers. They don’t help to maintain the thing, but if they fail you have a skew. At the bottom are the control programs, then the various service routines. Further up we have what I call middleware.

This is because no matter how good the manufacturer’s software for items like file handling it is just not suitable; it’s either inefficient or inappropriate. We usually have to rewrite the file handling processes, the initial message analysis and above all the real-time schedulers, because in this type of situation the application programs interact and the manufacturers, software tends to throw them off at the drop of a hat, which is somewhat embarrassing. On the top you have a whole chain of application programs.

The point about this pyramid is that it is terribly sensitive to change in the underlying software such that the new version does not contain the old as a subset. It becomes very expensive to maintain these systems and to extend them while keeping them live.

A. d'Agapeyeff emphasized the delicate balance within this pyramid, noting how sensitive it is to changes in the underlying software that do not preserve backward compatibility. He also warned against danger of over-generalized software too often unsuitable to any practical need:

In aiming at too many objectives the higher-level languages have, perhaps, proved to be useless to the layman, too complex for the novice and too restricted for the expert.

Despite improvements in general-purpose file handling and other advancements since d’Agapeyeff's time, the essence of his observations remains relevant.

There is still a big gap between low-level infrastructure, today encapsulated in an Operating System, like Linux, and the needs of final applications. The Operating System layer reflects and simplifies access to hardware capabilities, which are common for almost all applications.

Higher-level infrastructure needs, however, vary between different groups of applications: some prioritize minimizing the operational cost, some others - speed of development, and others - highly tightened security.

Different implementations of the Middleware layer are intended to fill up this gap and to provide domain-neutral services that are better tailored to the non-functional requirements of various groups of applications.

This consideration also explains why it’s always preferable to keep the core language, aka Winglang, and its standard library relatively small and stable, leaving more variability to be addressed by these intermediate Middleware layers.

Patterns, Frameworks, and Middleware

The middleware definition was refined in the “Patterns, Frameworks, and Middleware: Their Synergistic Relationships” paper, published in 2003 by Douglas C. Schmidt and Frank Buschmann. Here, they define middleware as:

software that can significantly increase reuse by providing readily usable, standard solutions to common programming tasks, such as persistent storage, (de)marshaling, message buffering and queueing, request demultiplexing, and concurrency control. Developers who use middleware can therefore focus primarily on application-oriented topics, such as business logic, rather than wrestling with tedious and error-prone details associated with programming infrastructure software using lower-level OS APIs and mechanisms.

To understand the interplay between Design Patterns, Frameworks and Middleware, let’s start with formal definitions derived from the “Patterns, Frameworks, and Middleware: Their Synergistic Relationships” paper Abstract:

Patterns codify reusable design expertise that provides time-proven solutions to commonly occurring software problems that arise in particular contexts and domains.

Frameworks provide both a reusable product-line architecture – guided by patterns – for a family of related applications and an integrated set of collaborating components that implement concrete realizations of the architecture.

Middleware is reusable software that leverages patterns and frameworks to bridge the gap between the functional requirements of applications and the underlying operating systems, network protocol stacks, and databases.

In other words, Middleware is implemented in the form of one or more Frameworks, which in turn apply several Design Patterns to achieve their goals including future extensibility. Exactly this combination, when implemented correctly, ensures Middleware's ability to flexibly address the infrastructure needs of large yet distinct groups of applications.

Let’s take a closer look at the definitions of each element presented above.

Design Patterns

In the realm of software engineering, a Software Design Pattern is understood as a generalized, reusable blueprint for addressing frequent challenges encountered in software design. As defined by Wikipedia:

In software engineering, a software design pattern is a general, reusable solution to a commonly occurring problem within a given context in software design. It is not a finished design that can be transformed directly into source or machine code. Rather, it is a description or template for how to solve a problem that can be used in many different situations. Design patterns are formalized best practices that the programmer can use to solve common problems when designing an application or system.

Sometimes, the term Architectural Pattern is used to distinguish high-level software architecture decisions from lower-level, implementation-oriented Design Patterns, as defined in Wikipedia:

An architectural pattern is a general, reusable resolution to a commonly occurring problem in software architecture within a given context. The architectural patterns address various issues in software engineering, such as computer hardware performance limitations, high availability and minimization of a business risk. Some architectural patterns have been implemented within software frameworks.

It is essential to differentiate Architectural and Design Patterns from their implementations in specific software projects. While an Architectural or Design Pattern provides an initial idea for solution, its implementation may involve a combination of several patterns, tailored to the unique requirements and nuances of the project at hand.

Architectural Patterns, such as Pipe-and-Filters, and Design Patterns, such as the Decorator, are not only about solving problems in code. They also serve as a common language among architects and developers, facilitating more straightforward communication about software structure and design choices. They are also invaluable tools for analyzing existing solutions, as we will see later.

Software Frameworks

In the domain of computer programming, a Software Framework represents a sophisticated form of abstraction, designed to standardize the development process by offering a reusable set of libraries or tools. As defined by Wikipedia:

In computer programming, a software framework is an abstraction in which software, providing generic functionality, can be selectively changed by additional user-written code, thus providing application-specific software.

It provides a standard way to build and deploy applications and is a universal, reusable software environment that provides particular functionality as part of a larger software platform to facilitate the development of software applications, products and solutions.

In other words, a Software Framework is an evolved Software Library that employs the principle of inversion of control. This means the framework, rather than the user's application, takes charge of the control flow. The application-specific code is then integrated through callbacks or plugins, which the framework's core logic invokes as needed.

Utilizing a Software Framework as the foundational layer for integrating domain-specific code with the underlying infrastructure allows developers to significantly decrease the development time and effort for complex software applications. Frameworks facilitate adherence to established coding standards and patterns, resulting in more maintainable, scalable, and secure code.

Nonetheless, it's crucial to follow the Clean Architecture guidelines, which mandate that domain-specific code remains decoupled and independent from any framework to preserve its ability to evolve independently of any infrastructure. Therefore, an ideal Software Framework should support plugging into it a pure domain code without any modification.

Middleware

The Middleware is defined by Wikipedia as follows:

Middleware is a type of computer software program that provides services to software applications beyond those available from the operating system. It can be described as "software glue".

Middleware in the context of distributed applications is software that provides services beyond those provided by the operating system to enable the various components of a distributed system to communicate and manage data. Middleware supports and simplifies complex distributed applications. It includes web servers, application servers, messaging and similar tools that support application development and delivery. Middleware is especially integral to modern information technology based on XML, SOAP, Web services, and service-oriented architecture.

Middleware, however, is not a monolithic entity but is rather composed of several distinct layers as we shall see in the next section.

Middleware Layers

Below is an illustrative diagram portraying Middleware as a stack of such layers, each with its specialized function, as suggested in the Schmidt and Buchman paper:

Fig 2: Middleware Layers

Fig 2: Middleware Layers in Context

Layered Architecture Clarified

To appreciate the significance of this layered structure, a good understanding of the very concept of Layered Architecture is essential—a concept too often misunderstood completely and confused with the Multitier Architecture, deviating significantly from the original principles laid out by E.W. Dijkstra.

At the “1968 NATO Software Engineering Conference,” E.W. Dijkstra presented a paper titled “Complexity Controlled by Hierarchical Ordering of Function and Variability” where he stated:

We conceive an ordered sequence of machines: A[0], A[1], ... A[n], where A[0] is the given hardware machine and where the software of layer i transforms machine A[i] into A[i+1]. The software of layer i is defined in terms of machine A[i], it is to be executed by machine A[i], the software of layer i uses machine A[i] to make machine A[i+1].

In other words, in a correctly organized Layered Architecture, the higher-level virtual machine is implemented in terms of the lower-level virtual machine. Within this series, we will come back to this powerful technique over and over again.

Back to the Middleware Layers

Right beneath the Applications layer resides the Domain-Specific Middleware Services layer, a notion deserving a separate discussion within the broader framework of Domain-Driven Design.

Within this context, however, we are more interested in the Distribution Middleware layer, which serves as the intermediary between Host Infrastructure Middleware within a single "box" and the Common Middleware Services layer which operates across a distributed system's architecture.

As stated in the paper:

Common middleware services augment distribution middleware by defining higher-level domain-independent reusable services that allow application developers to concentrate on programming business logic.

With this understanding, we can now place Winglang Middleware within the Middleware Services layer enabling the implementation of Domain-Specific Middleware Services in terms of its primitives.

To complete the picture, we need more quotes from the “Patterns, Frameworks, and Middleware: Their Synergistic Relationships” article mapped onto the modern cloud infrastructure elements.

Host Infrastructure Middleware

Here is how it’s defined in the paper:

Host infrastructure middleware encapsulates and enhances native OS mechanisms to create reusable event demultiplexing, interprocess communication, concurrency, and synchronization objects, such as reactors; acceptors, connectors, and service handlers; monitor objects; active objects; and service configurators. By encapsulating the peculiarities of particular operating systems, these reusable objects help eliminate many tedious, error-prone, and non-portable aspects of developing and maintaining application software via low-level OS programming APIs, such as Sockets or POSIX pthreads.

In the AWS environment, general-purpose virtualization services such as AWS EC2 (computer), AWS VPC (network), and AWS EBS (storage) play this role.

On the other hand, when speaking about the AWS Lambda execution environment, we may identify AWS Firecracker, AWS Lambda standard and custom Runtimes, AWS Lambda Extensions, and AWS Lambda Layers as also belonging to this category.

Distribution Middleware

Here is how it’s defined in the paper:

Distribution middleware defines higher-level distributed programming models whose reusable APIs and objects automate and extend the native OS mechanisms encapsulated by host infrastructure middleware.

Distribution middleware enables clients to program applications by invoking operations on target objects without hard-coding dependencies on their location, programming language, OS platform, communication protocols and interconnects, and hardware.

Within the AWS environment, fully managed API, Storage, and Messaging services such as AWS API Gateway, AWS SQS, AWS SNS, AWS S3, and DynamoDB would fit naturally into this category.

Common Middleware Services

Here is how it’s defined in the paper:

Common middleware services augment distribution middleware by defining higher-level domain-independent reusable services that allow application developers to concentrate on programming business logic, without the need to write the “plumbing” code required to develop distributed applications via lower-level middleware directly.

For example, common middleware service providers bundle transactional behavior, security, and database connection pooling and threading into reusable components, so that application developers no longer need to write code that handles these tasks.

Whereas distribution middleware focuses largely on managing end-system resources in support of an object-oriented distributed programming model, common middleware services focus on allocating, scheduling, and coordinating various resources throughout a distributed system using a component programming and scripting model.

Developers can reuse these component services to manage global resources and perform common distribution tasks that would otherwise be implemented in an ad hoc manner within each application. The form and content of these services will continue to evolve as the requirements on the applications being constructed expand.

Formally speaking, Winglang, its Standard Library, and its Extended Libraries collectively constitute Common middleware services built on the top of the cloud platform Distribution Middleware and its corresponding lower-level Common middleware services represented by the cloud platform SDK for JavaScript and various Infrastructure as Code tools, such as AWS CDK or Terraform.

With Winglang Middleware we are looking for a higher level of abstraction built in terms of the core language and its library and facilitating the development of production-grade Domain-specific middleware services and applications on top of it.

Domain-Specific Middleware Services

Here is how it’s defined in the paper:

Domain-specific middleware services are tailored to the requirements of particular domains, such as telecom, e-commerce, health care, process automation, or aerospace. Unlike the other three middleware layers discussed above that provide broadly reusable “horizontal” mechanisms and services, domain-specific middleware services are targeted at “vertical” markets and product-line architectures. Since they embody knowledge of a domain, moreover, reusable domain-specific middleware services have the most potential to increase the quality and decrease the cycle-time and effort required to develop particular types of application software.

To sum up, the Winglang Middleware objective is to continue the trend of the Winglang compiler and its standard library to make developing Domain-specific middleware services less difficult.

Cloud Middleware State of Affairs

Applying the terminology introduced above, the current state of affairs with AWS cloud Middleware could be visualized as follows:

Fig 3: Cloud Middleware State of Affairs

We will look at three leading Middleware Frameworks for AWS:

  1. Middy (TypeScript)
  2. Power Tools for AWS Lambda (Python, TypeScript, Java, and .NET)
  3. Lambda Middleware

Middy

If we dive into the Middy Documentation we will find that it positions itself as a middleware engine, which is correct if we recall that very often Frameworks, which Middy is, are called Engines. However, it later claims that “… like generic web frameworks (fastify, hapi, express, etc.), this problem has been solved using the middleware pattern.” This is, as we understand now, complete nonsense. If we dive into the Middy Documentation further, we will find the following picture:

Fig 4: Middy

Now, we realize that what Middy calls “middleware” is a particular implementation of the Pipe-and-Filters Architecture Pattern via the Decorator Design Pattern. The latter should not be confused with TypeScript Decorators. In other words, Middy decorators are assembled into a pipeline each one performing certain operations before and/or after an HTTP request handling.

Perhaps, the main culprit of this confusion is the expressjs Framework Guide usage of titles like “Writing Middleware” and “Using Middleware” even though it internally uses the term middleware function, which is correct.

Middy comes with an impressive list of official middleware decorator plugins plus a long list of 3rd party middleware decorator plugins.

Power Tools for AWS Lambda

Here, the basic building blocks are called Features, which in many cases are Adapters of lower-level SDK functions. The list of features for different languages varies with the Python version to have the most comprehensive one. Features could be attached to Lambda Handlers using language decorators, used manually, or, in the case of TypeScript, using Middy. The term middleware pops up here and there and always means some decorator.

Lambda Middleware

This one is also an implementation of the Pipe-and-Filters Architecture Pattern via the Decorator Design Pattern. Unlike Middy, individual decorators are combined in a pipeline using a special Compose decorator effectively applying the Composite Design Pattern.

Limitations of existing solutions

Apart from using the incorrect terminology, all three frameworks have certain limitations in common, as follows:

  1. The confusing sequence of operation of multiple Decorators. When more than one decorator is defined, the sequence of before operations is in the order of decorators, but the the sequence of after operations is in reverse order. With a long list of decorators that might be a source of serious confusion or even a conflict.

  2. Reliance of environment variables. Control over the operation of particular adapters (e.g. Logger) solely relies on environment variables. To make a change, one will need to redeploy the Lambda Function.

  3. A single list of decorators with some limited control in runtime. There is only one list of decorators per Lambda Function and, if some decorators need to be excluded and replaced depending on the deployment target or run-time environment, a run-time check needs to be performed (look, for example, at how Tracer behavior is controlled in Power Tools for AWS Lambda). This introduces unnecessary run-time overhead and enlarges the potential security attack surface.

  4. Lack of support for higher-level crosscut specifications. All middleware decorators are specified for individual Lambda functions. Common specifications at the organization, organization unit, account, or service levels will require some handmade custom solutions.

  5. Too narrow interpretation of Middleware as a linear implementation of Pipe-and-Filers and Decorator design patterns. Power Tools for AWS Lambda makes it slightly better by introducing its Features, also called Utilities, such as Logger, first and corresponding decorators second. Middy, on the other hand, treats everything as a decorator. In both cases, the decorators are stacked in one linear sequence, such that retrieving two parameters, one from the Secrets Manager and another from the AppConfig, cannot be performed in parallel while state-of-the-art pipeline builders, such as Marble.js and Async.js, support significantly more advanced control forms.

For Winglang Common Services Middleware Framework (we can now use the correct full name) this list of limitations will serve as a call for action to look for pragmatic ways to overcome these limitations.

Winglang Middleware Direction

Following the “Patterns, Frameworks, and Middleware: Their Synergistic Relationships” article middleware layers taxonomy, the Winglang Common Middleware Services Framework is positioned as follows:

Fig 5: Winglang Middlware Layer

In the diagram above, the Winglang Middleware Layer, code name Winglang MW, is positioned as an upper sub-layer of Common Middleware Services, built on the top of the Winglang as a representative of the Infrastructure-from-Code solution, which in turn is built on the top of the cloud-specific SDK and IaC solutions providing convenient access to the cloud Distribution Middleware.

From the feature set perspective, the Winglang MW is expected

  1. To be on par with leading middleware frameworks such
    1. Middy (TypeScript)
    2. Power Tools for AWS Lambda (Python, TypeScript, Java, and .NET)
    3. Lambda Middleware
  2. In addition, to provide support for leading open standards such as
    1. OpenID
    2. Open Telemetry
    3. OAuth 2.0
    4. Async API
    5. Cloud Events
  3. To provide built-in support for cross-cut middleware specifications at different levels above individual cloud functions
  4. To support run-time fine-tuning of individual feature parameters (e.g. logging level) without corresponding cloud resources redeployment

Different implementations of Winglang MW will vary in efficiency, ease of use (e.g. middleware pipeline configuration), flexibility, and supplementary tooling such as automatic code generation.

At the current stage, any premature conversion towards a single solution will be detrimental to the Winglang ecosystem evolution, and running multiple experiments in parallel would be more beneficial. Once certain middleware features prove themselves, they might be incorporated into the Winglang core, but it’s advisable not to rush too fast.

For the same reason, I intentionally called this section Directions rather than Requirements or Problem Statement. I have only a general sense of the desirable direction to proceed. Making things too specific could lead to some serendipitous alternatives being missed. I can, however, specify some constraints, as follows:

  1. Do not count on advanced Winglang features, such as Generics, to come. Generics may significantly complicate the language syntax and too often are introduced before a clear understanding of how much sophistication is required. Also, at the early stages of exploration, the lack of Generics support could be compensated by switching to a general-purpose data type, such as Json, or code generators, including the “C” macros.
  2. Stick with Winglang and switch to TypeScript for implementing low-level extensions only. As a new language, Winglang lacks features taken for granted in mainstream languages and therefore requires some faith to get a fair chance to write as much code as possible, even if it is slightly less convenient. This is the only way for a new programming language to evolve.
  3. If the development of CLI tools is required, prefer TypeScript over other languages such as Python. I already have the TypeScript toolchain installed on my desktop with all dependencies resolved. It’s always better to limit the number of moving parts in the system to the absolute minimum.
  4. Limit Winglang middleware implementation to a single process of a Cloud Function. Out-of-proc capabilities, such as AWS Lambda Extensions, can improve overall system performance, security, and reuse (see, for example, this blog post). However, they are not currently supported by Winglang out of the box. Also, utilizing such advanced capabilities will increase the system's complexity while contributing little, if any, at the semantic level. Exploring this direction can be postponed to later stages.

What’s Next?

This publication was completely devoted to clarifying the concept of Middleware, its position within the cloud software system stack, and defining a general direction for developing one or more Winglang Middleware Frameworks.

I plan to devote the next Part Two of this series to exploring different options for implementing the Pipe-and-Filters Pattern in Middleware and after that to start building individual utilities and corresponding filters one by one.

It’s a rare opportunity that one does not encounter every day to revise the generic software infrastructure elements from the first principles and to explore the most suitable ways of realizing these principles on the leading modern cloud platforms. If you are interested in taking part in this journey, drop me a line.