A producer / distributed consumer library for .NET applications. Targets .NET 4.8, .NET 8.0, .NET 10.0, and .NET Standard 2.0.
High-level features:
- Queue / de-queue POCOs for distributed processing
- Queue / process LINQ statements (compiled or dynamic, expressed as a string)
- Re-occurring job scheduler
See the Wiki for in-depth documentation.
Base
| Package | NuGet |
|---|---|
| DotNetWorkQueue |  |
Transports
| Package | NuGet |
|---|---|
| DotNetWorkQueue.Transport.Redis |  |
| DotNetWorkQueue.Transport.SqlServer |  |
| DotNetWorkQueue.Transport.SQLite |  |
| DotNetWorkQueue.Transport.PostgreSQL |  |
| DotNetWorkQueue.Transport.LiteDb |  |
Metrics
| Package | NuGet |
|---|---|
| DotNetWorkQueue.AppMetrics |  |
.NET Standard 2.0, 8.0, and 10.0 are missing the following features compared to the full framework version:
- No support for aborting threads when stopping consumer queues
- No support for dynamic LINQ statements
Producers
| Transport | Sample |
|---|---|
| SQL Server | SQLServerProducer/Program.cs |
| SQLite | SQLiteProducer/Program.cs |
| Redis | RedisProducer/Program.cs |
| PostgreSQL | PostgreSQLProducer/Program.cs |
| LiteDb | LiteDbProducer/Program.cs |
Consumers
| Transport | Sample |
|---|---|
| SQL Server | SQLServerConsumer/Program.cs |
| SQLite | SQLiteConsumer/Program.cs |
| Redis | RedisConsumer/Program.cs |
| PostgreSQL | PostGreSQLConsumer/Program.cs |
| LiteDb | LiteDbConsumer/Program.cs |
You can queue LINQ expressions to be executed instead of POCOs. This makes producers and consumers generic — they no longer need to be message-specific. The examples below are not transport-specific and assume any queue creation steps have already been performed.
Note: It is possible for a producer to queue work that a consumer cannot process. In order for a consumer to execute a LINQ statement, all types must be resolvable. For dynamic statements, it is also possible to queue work that doesn't compile due to syntax errors — this won't be discovered until the consumer dequeues the work.
Note: When passing
messageorworkerNotificationas arguments to dynamic LINQ, you must cast them, as the internal compiler treats them asobject. This is not necessary when using standard LINQ expressions.
// Cast types when using dynamic LINQ:
(IReceivedMessage<MessageExpression>) message
(IWorkerNotification) workerNotificationWhen passing value types, you will need to parse them inline. For example, a Guid and an int can be embedded in string literals and parsed using built-in .NET methods:
var id = Guid.NewGuid();
var runTime = 200;
$"(message, workerNotification) => StandardTesting.Run(new Guid(\"{id}\"), int.Parse(\"{runTime}\"))"This produces a LINQ expression that can be compiled and executed by the consumer, provided it can resolve all referenced types.
The consumer is generic and can process any LINQ expression, but it must be able to resolve all types the expression references. You may need to wire up an assembly resolver if your DLLs cannot be located automatically.
No sandboxing or checking for risky commands is performed. For example, the following statement will cause the consumer host to exit:
"(message, workerNotification) => Environment.Exit(0)"If configuration files define dynamic LINQ statements, or if you cannot fully trust the producer, consider running the consumer in an application domain sandbox. Without that, the only protection against destructive commands is O/S user permissions:
"(message, workerNotification) => System.IO.Directory.Delete(@\"C:\\Windows\\\", true)"Jobs may be scheduled using Schyntax format. The scheduler and consumers are separate — schedulers only queue work, while standard LINQ consumers handle processing.
Any LINQ statement supported by a LINQ producer can be scheduled via the scheduler.
Multiple schedulers sharing the same schedule can be run for redundancy, but it is important that the clocks on all machines are in sync, or that the same time provider is injected into both schedulers and consumers. See the Wiki for more details.
Note: Using multiple machines with out-of-sync clocks may produce unexpected results. Server-based transports often provide solutions for this. See the Wiki.
See Schyntax for the event scheduling format.
The scheduler and its container must remain in scope for as long as you are scheduling work. Disposing or allowing the scheduler to go out of scope will stop all queuing.
To consume and process scheduled jobs, use a LINQ Consumer:
SQLiteSchedulerConsumer/Program.cs
You'll need Visual Studio 2022/2026 (any edition) and the .NET Core 2.0 / 8.0 / 10.0 SDKs installed.
All references are either on NuGet or in the \lib folder. Building from Visual Studio should restore all required files automatically.
Copyright © 2015–2026 Brian Lehnen
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 2.1 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this program. If not, see http://www.gnu.org/licenses/.
- LibLog
- NetFX-Guard
- SimpleInjector
- Microsoft.IO.RecyclableMemoryStream
- Newtonsoft.Json
- JpLabs.DynamicCode
- JsonNet.PrivateSetterContractResolvers
- Expression-JSON-Serializer
- Schtick
- Schyntax
- Polly
(No additional dependencies)
