Alternative To Hangfire

Introduction

Ask a C# developer about a background job engine library and chances are the developer will mention Hangfire. Hangfire has been around for quite a while, and it generally seems well liked by the C# community. It has been in development as far back as 2012 and is still actively maintained today. You can find its main and secondary repositories on GitHub; however, the creator also has a dedicated website for the product.

In this article, we are going to explore the differences between Didact and the Hangfire library and show why Didact might be the alternative solution you and your team are looking for.

Open Source Licensing

Hangfire Licensing

First and foremost, the Hangfire is open source licensed under the LGPL3.0 open source license. I wrote a brief article on IndieHackers.com explaining a general guideline to use for open source licensing and why the GNU licenses are often favorable for open source creators that make platforms vs. libraries - check it out if you're thinking of building an open source application.

The LGPL3.0 allows you to use Hangfire privately and publicly for commercial use, but it has some copyleft restrictions typical of GNU licenses that de-incentivize corporations from redistributing modified versions of the open source work as closed source or without contribution. However, Hangfire has several pricing plans on its website, and you can purchase elevated, commercial-friendly licenses at an annual price that permit more advanced usage and code modifications to the Hangfire library itself.

One thing to stress here is that, again, Hangfire is a library to incorporate into your larger software work, it is not a standalone application. We will come back to that in a moment.

Hangfire is a library that is meant to be incorporated into your larger software work, so it's under the LGPL3.0.

Didact Licensing

Similarly, Didact is also open source licensed. However, Didact is licensed under the AGPLv3 rather than the LGPL3.0. This has a very similar set of terms to the LGPL3.0 such as allowing both private and public commercial use. However, with the AGPLv3, offering the software product through a network - such as a cloud service - counts as distribution, so the same copyleft terms apply: if you modify the original work and redistribute your changes, then you must open source your modified version.

Coming back to an earlier point, the main reason that Didact is licensed under AGPLv3 is because Didact is a complete, pre-built, standalone application; I am building it in such a way that offering it as a cloud service is a very natural use case - which is exactly what I'm planning to do to help monetize and sustain the project. So AGPLv3 makes the most sense for the project.

Didact is a standalone application, separate from your larger software work. Additionally, I am offering it as a cloud service, so AGPLv3 is the best option.

Licensing Concerns

If you're concerned about the difference in licensing for Didact, don't worry! Chances are, you actually have less to worry about using Didact than using Hangfire.

Why? Because although the AGPLv3 license for Didact is, technically speaking, more copyleft restrictive than the LGPL3.0 license, Didact is a standalone application, not a library to be incorporated into a larger software work. It has a pre-built, completely decoupled, separate repository for the REST API engine, the web dashboard, the console application, and the DLL nuget packages for your Flows. Even when it comes to defining your Flows, Didact requires you to setup a separate class library project that will exist as an indepenent entity in your codebase.

This means that any part of Didact will most likely exist outside and separate from your larger software work, so the GNU copyleft restrictions will not apply; in other words, you won't be forced to open source your work.

TL;DR; You should be fine. But if you're concerned, feel free to reach out!

Flows

The core component of processing your background jobs in Didact is the Flow. A Flow is a class that you define in your central Flow class library project by implementing several core interfaces, such as IFlow.

For example, see the Flow below:

public class SomeFlow : IFlow
{
    // Constructor setup...

    // Other configurations...

    // The actual job definition
    public async Task ExecuteAsync()
    {
        DoSomething();
        DoSomethingElse();
        // ...
    }
}

The Flow is where you define what your background job or automation. Here you can configure input variables if you need any, triggers, schedules, and more.

To execute your background job, you must implement an asynchronous method called ExecuteAsync(). This method will define what your Flow actually does, and it runs asynchronously to allow for maximum performance in Didact Engine. Unlike directed acyclic graphs (DAGs) in Apache Airflow, Flows are not restricted to an acyclic chain of commands; ExecuteAsync() is just a method, so you can code whatever you want inside of it. When Didact Engine instantiates your Flow, it will know to execute this method to actually run the Flow.

This differs from background jobs in Hangfire which are defined in single methods like BackgroundJob.Enqueue() or RecurringJob.AddOrUpdate().

Since, at its core, a Flow is simply a class that implements a few interfaces, it offers all of the flexibility and power that a standard C# class offers while enforcing a strong decoupling from other code, making it a powerful alternative to the Hangfire background job methods.

Execution Blocks

Didact offers a highly-granular level of execution tracking at the method level. It does this via a family of delegate execution "blocks" in the DidactCore nuget package. These execution blocks allow you to run any delegate inside of them and are offered via a set of generics; the generics make Didact a much stronger alternative to similar tools in Python like Apache Airflow or Prefect because it enforces type safety for inputs and outputs at the method level.

Because the blocks accept delegates, you can run a variety of code types inside of them, such as:

• Lambda functions
• Methods from your class library project
• Delegates, Actions, and Funcs

These blocks expose an advanced set of configurations at the method level such as retry policies, execution timeouts, inputs, outputs, start times, end times, and more. What's better, you can use these configurations by way of an easy-to-use, optional fluent API syntax.

For example, consider the execution blocks block1 and block2 being used in SomeFlow below:

using DidactCore;

public class SomeFlow : IFlow
{
    // Constructor setup...

    // Other configurations...

    // The actual job definition
    public async Task ExecuteAsync()
    {
        var block1 = new ActionBlock(AMethodFromSomewhere())
            .Timeout(5)
            .Retry(5);

        var block2 = new TransformerBlock<string, int>(ConvertStringToInteger())
            .Timeout(1);

        await block1.ExecuteAsync();
        await block2.ExecuteAsync();
    }
}

Ideally, when you create a Flow in Didact, you are making it out of a chain of execution blocks. Your logic should be encapsulated inside of these blocks for maximum tracking and logging. However, these blocks are not as rigid as a DAG in Apache Airflow, so you can be very flexible with how you execute them.

This differs significantly from what is offered in Hangfire. If you compare Didact to Hangfire, a Flow in Didact is analogous to a BackgroundJob or RecurringJob in Hangfire. However, out of the box, Hangfire does not offer atomic execution tracing like Didact does via its block family. You can see that, with Didact, you have a far greater level of tracing and execution control.

Class Library Project

The key component to Didact processing your background jobs is by executing your Flows in your central class library project. In essence, this is a separate class library project for .NET - meaning it produces .dll files, not an executable - that houses all of your Flow classes and their respective configurations.

Hangfire - as a library - is designed to be run inside of a pre-existing application. While this may not be an issue for some, this setup can become disorganized and difficult to manage over time. Your entire background job processing is littered inside of your application codebase.

Didact, on the other hand, enforces a strong separation of concerns and code isolation from any other application since all of your Flows live inside of the class library project. This completely seperates your background job processing from your applications and offers it as a standalone, configurable platform in and of itself since Didact Engine executes your Flows. I believe this makes several aspects of background job processing, such as scaling, much easier for you to manage since you do not have to duplicate an entire other application or litter configurations within the main codebase.

Flow Triggers

Didact will offer several different trigger types for activating Flows.

Scheduled Flows

First and foremost, Flows have a scheduling option in their fluent API configurations. Similar to Hangfire, you can choose a recurring schedule with optional settings such as an end date and a CRON expression.

Dashboard Trigger

Also similar to Hangfire, you will be able to execute a Flow on an adhoc basis from inside of the Didact web dashboard via the user interface.

REST Endpoint Trigger

However, unlike Hangfire, you will also have the option to launch Flows on an adhoc basis via a REST API endpoint. With a simple HTTP request and a few required parameters like the FlowId, you will be able to execute Flows on an event-driven basis for dynamic, just-in-time processing.

Modern Web Dashboard

A beautiful user interface can go a very long way, so I'm aiming to provide a simple, beautiful, and robust web dashboard in Didact UI. It is built with Tailwind CSS and Flowbite, a powerful Tailwind component library, and I plan to use Observable Plot to create pixel perfect charts and visualizations. The web dashboard is a completely separate VueJS project that you can find in the Didact UI repository.

Dashboard Data

In the web dashboard, you will be able to access a variety of information, such as:

• Flow execution metadata
• Execution block logs
• Execution metrics (successes, failures, retries, cancellations, etc.)
• Didact Engine configurations
• Secrets manager
• And more.

OpenAPI Swagger File

You will also be able to view all of the REST endpoints in Didact Engine in an embedded Swagger doc that is auto-generated from Visual Studio, making API exploration and discovery very easy.

Reporting

I also plan to offer some basic reporting for Flows and Flow executions. I will likely offer advanced reporting in a higher paid tier later on. As far as I am aware, Hangfire has minimal reporting capabilities in the web dashboard beyond basic Job runs.

Persistent Storage

Hangfire Storage

Hangfire's free tier offers two persistent storage options:

• In-memory (preview)
• SQL Server

However, you can purchase a paid Hangfire plan to unlock Redis storage. Additionally, the open source community behind Hangfire has developed a number of additional persistent storage options such as MySQL, PostgreSQL, and others. You can find a full list of them on the Hangfire site.

Didact Storage

Didact similarly offers a default persistent storage option of SQL Server. However, this will be the only storage option available upon release. After release, I plan to offer a PostgreSQL storage option for Didact - if the community does not build it first - to offer more flexible storage options.

Why So Few Storage Options?

At the time of this writing, I plan to utilize both the standard ORM Entity Framework Core and the micro ORM Dapper in Didact Engine with a strong emphasis on Dapper. I personally enjoy writing SQL queries, and I want the absolute maximum query performance that I can squeeze out of Didact Engine. As a result, many parts of the persistent storage repositories will not be as easy as swapping out versus if I was only using Entity Framework Core since I will have SQL queries specific to certain database engines.

I think that Dapper and the use of IMemoryCache's will show that using a SQL-based queue is a powerful form of background job processing.

What about Redis?

I have already had a few different people ask me about using Redis storage due to its incredibly low latency. My opinion is that Redis is an excellent technology for a variety of use cases, including basic queues. However, I am not focusing on Redis at the moment for a few different reasons:

• Redis requires more advanced infrastructure setup when we already have the web dashboard, the REST API engine, and the Flow class library project for the default use cases of Didact.
• I am expecting that many .NET shops will want to pick up Didact, and they will naturally already have SQL Server or Azure SQL instances setup in their on-prem or cloud environments.
• The queues, flow storage, and job execution are setup under a very specific data model that is optimized for relational database storage.
• Didact Engine will leverage several IMemoryCache objects to allow for low latency data fetching on various pages of the web dashboard. For the moment, I am confident that these IMemoryCache objects will be sufficient for Didact's needs.

If the community finds out that IMemoryCache's are not powerful enough for Didact's caching needs and we need a more advanced caching solution, then I will be happy to consider adding a Redis component. However, even if Redis is added to the central tech stack for Didact, I don't plan to use it as a primary database store; I will use it strictly as a cache.

Multitenancy

Hangfire does not have a concept of multitenancy. There is no central organization or tenant table within the Hangfire data model. This can be a complicated issue to solve for businesses looking to utilize Hangfire jobs for various customer needs. Especially for businesses with a large number of customers, it can be cumbersome and time-consuming to develop workarounds for separating out customers. In truth, you really need a proper primary key data model, row-level security, or some other partitioning method.

On the other hand, multitenancy is a core part of the Didact data model. Didact has a central Organization table with a primary key ID, and that ID is found as a foreign key in almost every other table in the data model. This adds a powerful incentive for cloud hosting and ensures that you can easily partition Flows, Executions, and more by customer or tenant.

Queues

Queue Types

Generally, I consider two primary queue types (taking inspiration from AWS SQS):

• Standard (Best Effort) Queues
• FIFO Queues

Standard queues allow for high throughput and do not guarantee strict ordering; they will give it their best effort, but they will not bottlebeck other queued items from being processed if an upstream item fails. For example, if your queueu is a SQL table with an integer primary key, and you need to process IDs 6 - 10, then ID 9 and ID 10 will still process out of the queue even if ID 8 fails.

FIFO queues, on the other hand, are queues constrained by lower throughput, but they guarantee order execution. Going back to the same example, if you have a FIFO queue that needs to process IDs 6- 10, then ID 9 and ID 10 will not process out of the queue if ID 8 fails; instead, ID 8 will need to be retried until it succeeds or moved to a dead letter queue.

Didact's Queue Offerings

Hangfire only offers standard (best effort) queues, but it does not offer FIFO queues. Didact, on the other hand, supports both queue types in its data model. In the Didact database, you will find both a StandardQueue and a FifoQueue table.

Many Hangfire users might find this very desirable, as I've seen various examples online of Hangfire users asking for FIFO queue support. This can become increasingly important the larger your background job requirements become. For example, if you have many customers that you want to easily distinguish from each other, or if you want to separate background jobs for customers from your company's internal background jobs, then this could be a high-impact feature for you and your team.

Multitenancy Queues

Another nice feature of Didact's queues are that they have a foreign key to the Organization table in them, so they naturally support multitenancy. This should make separating queues for you and your customers much easier! Again, Hangfire does not have a concept of multitenancy in its data model.

Multiple Queues

Additionally, you can define mulitple queues per tenant. This allows a very granular partitioning of Flows. Hangfire also offers multiple queues in its data model but without the multitenancy support.

Dependency Injection

Most of us can agree that one of the best parts of .NET Core and the new .NET Standard is the dependency injection system it provides. Inversion of control becomes a much simpler pattern to implement when you use the built in DI system in new .NET projects.

I believe that you will find Didact's dependency injection much easier to work with than the dependency injection system in Hangfire after reading the details below.

Hangfire's Dependency Injection

Hangfire allows you to implement dependency injection, but it has a few caveats.

Hangfire requires you to pass services from the dependency injection container into its primary methods such as BackgroundJob.Enqueue by using generic overloads <>. If you need to use a DI service in your Hangfire job, then you need to specify the service in the generic overload.

For example, in the code snippet below, we are attempting to pass a service called SomeService from the dependency injection container into our background job:

BackgroundJob.Enqueue<SomeService>(x => Console.WriteLine(x.someProperty));

However, continuing with our example, there are some difficulties when registering the SomeService class. The generic overloads for Hangfire's methods only allow you to pass in one Type, not a list of Types, to your background job. So if you need a large number of dependencies in your job, you might have to bloat your SomeService class or compose SomeService from several smaller classes like SomeSmallerServiceA and SomeSmallerServiceB. Although you can make this work within your codebase, it might unintentionally restrict or bloat your dependency usages. See this Stack Overflow post for an example.

Additionally, users have commented that the DI system in Hangfire does not properly dispose of dependencies with lifetime management. See the comments from this Stack Overflow post for an example.

In my opinion, it's a case of a feature working, but I think it could be improved upon.

Didact's Dependency Injection

Contrastly, Didact has a much different approach to dependency injection. Please recall that, first and foremost, Didact is a standalone, pre-built application that exists outside of your main codebase. To define your background jobs - again, called Flows in Didact - you create a central class library project and define your jobs as classes in that class library project, not in your existing codebase.

Plug And Play

When the class library project is published, Didact Engine will identify your published .dll files, shut itself down, restart, and dynamically register your Flows and their dependencies in its pre-built dependency injection system.

You may be wondering, though, where and how you register dependencies and also how they are made available to Didact Engine? Again, this is not the same as Hangfire: Didact is built in a way such that you should not need to modify the Program.cs file in Didact Engine where the dependency injection system is; rather, this is supposed to work as a plug and play system.

Extension Method

Didact accomplishes this by introducing a static extension method that you need to implement in your central class library project. This method extends the .NET service collection IServiceCollection, and in this extension method, you define all of your dependencies to be used in your Flows.

For example, look at the extension method below which would be created as a class in your class library project:

public static class DidactDependencyInjection
{
    public static void AddDidactServices(this IServiceCollection services)
    {
        services.AddScoped<IServiceA, SomeServiceA>();
        services.AddTransient<IServiceB, SomeServiceB>();
        services.AddSingleton<ISomeSingleton, SomeSingleton>();
        
        // Register your other services...
    }
}

Then, in Didact Engine, there is a pre-configured chunk of code that looks for and executes this extension method at application startup.

For example, in the Program.cs file of Didact Engine:

services.AddTransient<InternalDidactServiceA>();
services.AddTransient<InternalDidactServiceB>();

// Register other internal services...

// Pre-configured code in Didact Engine that adds the extension method from your class library.
services.AddDidactServices();

Automatic Flow Injection

Following this example all the way through, then, after Didact Engine would execute the AddDidactServices() extension method, all of the services needed for your Flows are registered and available in the dependency injection system in Didact Engine. So whenever Didact Engine would need to create a new instance of one of your Flows, it would automatically inject the required dependencies into your Flow class - entirely as a plug and play, pre-configured dependency injection system.

Following the example to completion, if you had registered

• IServiceA
• IServiceB
• ISomeSingleton

in the AddDidactServices() method of your class library project, and if you have a Flow in your class library project called SomeFlow, you could define it as below and have Didact Engine automatically inject all of the required dependencies:

public class SomeFlow : IFlow
{
    private readonly IServiceA _serviceA;
    private readonly IServiceB _serviceB;
    private readonly ISomeSingleton _someSingleton;

    public SomeFlow(IServiceA serviceA, IServiceB serviceB, ISomeSingleton someSingleton)
    {
        _serviceA = serviceA;
        _serviceB = serviceB;
        _someSingleton = someSingleton;
    }

    // Other configurations...

    public async Task ExecuteAsync()
    {
        _serviceA.DoSomething();
        // ...
        _serviceB.DoSomethingElse();
        // ...
        _someSingleton.DoWhatever();
        // ...
    }
}

Dependency Injection Takeaways

Ok, deep breath: that was a lot of technical details. So what does this all mean, and what advantages does it offer over Hangfire's way of doing dependency injection?

Here's a few thoughts:

• Hangfire is a library to be added to a larger software work. You are meant to modify Program.cs files and manually wire up full DI systems "the normal way" when you are creating your own application.
• Hangfire only allows you to pass in one service from your dependency injection container at a time. This introduces some creative restrictions when trying to pass around dependencies into your background jobs.
• Didact is a standalone application, and it essentially offers the builtin .NET dependency injection system from Didact Engine as a sort of plug and play feature in your class library project.
• Didact does this by requiring you to create an IServiceCollection extension method - like AddDidactServices() - in your class library project. When you publish your class library, Didact Engine absorbs the .dll files and automatically executes the extension method on startup with pre-configured code that I've written.
• This allows total flexibility when adding dependencies to your Flows. You don't necessarily have to compose a plethora of custom classes in order to access dependencies. The one cost of this is registering your dependencies in the extension method of your class library project.

I think this is a nice evolution of the isolation of background jobs/Flows from similar tools in Python like Apache Airflow and Prefect while utilizing the incredibly powerful, builtin dependency injection system in .NET.

Summary

All in all, I think you will find Didact to be a powerful and capable alternative to the Hangfire library. Although it is not ready yet, I would love for you to submit your email and stay in touch. You will be the first to know when Didact is ready, and you'll receive discounts on all pricing plans in the future!

Come join the community and let's shake up the .NET world!