Building Custom Configuration Providers in .NET

One of the most powerful features of the .NET configuration system is its extensibility. While .NET provides many built-in configuration providers for common scenarios like JSON files, environment variables, and command-line arguments, you might encounter situations where you need a custom configuration source.

In this post, we’ll explore how to create custom configuration providers by examining a real-world example: the Configuration.Extensions.EnvironmentFile package, which allows .NET applications to use Unix-style environment files (.env) for configuration.

Why Custom Configuration Providers?

Before diving into implementation details, let’s consider some scenarios where custom configuration providers are useful:

• Supporting configuration formats not covered by built-in providers
• Retrieving configuration from specialized services or databases
• Implementing specific validation or transformation logic for configuration values
• Adding environment-specific configuration handling

The EnvironmentFile provider addresses the first scenario by supporting .env files, a popular format in the Unix world for configuring services.

Understanding the .NET Configuration Architecture

The .NET configuration system follows a provider pattern with these key components:

1. Configuration Sources: Define where configuration data comes from
2. Configuration Providers: Read from sources and provide data to the configuration system
3. Configuration Builder: Combines multiple providers into a unified configuration
4. Configuration Root: The final configuration that applications consume

Each provider implements specific interfaces from the Microsoft.Extensions.Configuration namespace. Let’s look at what we need to build a custom configuration provider.

Implementing a Custom Configuration Provider

Using the Configuration.Extensions.EnvironmentFile package as our example, we’ll walk through the essential components needed to create a custom configuration provider.

1. Define the Configuration Source

The configuration source defines where and how to get configuration data. It implements IConfigurationSource and creates the actual provider.

public class EnvironmentFileConfigurationSource : FileConfigurationSource
{
    public EnvironmentFileConfigurationSource()
    {
        Path = ".env";
    }

    internal string? Prefix { get; set; }
    internal bool RemoveWrappingQuotes { get; set; } = true;
    internal bool Trim { get; set; } = true;

    public override IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        EnsureDefaults(builder);
        return new EnvironmentFileConfigurationProvider(this, Trim, RemoveWrappingQuotes, Prefix);
    }
}

This source inherits from FileConfigurationSource because it works with files, which gives us file path handling and file provider functionality for free. The source defines specific options like prefix handling and quote formatting, then creates a provider instance in the Build method.

2. Create the Configuration Provider

The provider does the actual work of reading and parsing the configuration data:

internal class EnvironmentFileConfigurationProvider : FileConfigurationProvider
{
    private readonly bool _trim;
    private readonly bool _removeWrappingQuotes;
    private readonly string? _prefix;

    public override void Load(Stream stream)
    {
        Data = EnvironmentFileConfigurationParser.Parse(
            stream,
            _trim,
            _removeWrappingQuotes,
            _prefix
        );
    }

    internal EnvironmentFileConfigurationProvider(
        FileConfigurationSource source,
        bool trim,
        bool removeWrappingQuotes,
        string? prefix
    )
        : base(source)
    {
        _trim = trim;
        _removeWrappingQuotes = removeWrappingQuotes;
        _prefix = prefix;
    }
}

By inheriting from FileConfigurationProvider, we get file loading functionality automatically. We just need to override the Load method to parse the file content. In this case, the provider delegates the parsing to a separate class.

3. Implement the Parser

The parser contains the logic for extracting key-value pairs from the configuration source:

internal static class EnvironmentFileConfigurationParser
{
    public static Dictionary<string, string> Parse(
        Stream stream,
        bool trim,
        bool removeWrappingQuotes,
        string? prefix
    )
    {
        StreamReader reader = new(stream);

        IEnumerable<string> nonCommentLinesWithPropertyValues = reader
            .ReadToEnd()
            .Split(["\r\n", "\r", "\n"], StringSplitOptions.None)
            .Select(x => x.TrimStart())
            .Select(x => string.IsNullOrWhiteSpace(prefix) ? x : x.Replace(prefix, string.Empty))
            .Where(x => !x.StartsWith("#") && x.Contains("="));

        IEnumerable<KeyValuePair<string, string>>? configuration = nonCommentLinesWithPropertyValues
            .Select(ParseQuotes)
            .Select(x => RemoveCommentsAtTheEndAndTrimIfNecessary(x, trim))
            .Select(x => x.Split('='))
            .Select(x => new KeyValuePair<string, string>(
                x[0].Replace("__", ":"),
                string.Join("=", x.Skip(1))
            ));

        return configuration.ToDictionary(x => x.Key, x => x.Value);

        // Helper methods for processing values
        string ParseQuotes(string line)
        {
            if (!removeWrappingQuotes)
            {
                return line;
            }

            string[] parts = line.Split('=');
            line = string.Join("=", parts.Skip(1));
            return $"{parts[0]}={line.Trim('"')}";
        }

        static string RemoveCommentsAtTheEndAndTrimIfNecessary(string line, bool t)
        {
            return t ? line.Trim() : line;
        }
    }
}

This parser handles:

• Reading the file line by line
• Filtering out comment lines and empty lines
• Handling prefixes by removing them if specified
• Processing equals signs in values (preserving them after the first one)
• Converting double underscores to colons for hierarchical configuration
• Optionally trimming whitespace and removing quotes

4. Provide Extensions Methods for Easy Registration

To make the provider easy to use, we need extension methods for the IConfigurationBuilder:

public static class EnvironmentFileConfigurationExtensions
{
    public static IConfigurationBuilder AddEnvironmentFile(
        this IConfigurationBuilder builder,
        string fileName = ".env",
        bool trim = true,
        bool removeWrappingQuotes = true,
        string? prefix = null,
        bool reloadOnChange = false
    )
    {
        string? directory;
        if (!Path.IsPathRooted(fileName))
        {
            directory =
                builder.Properties.TryGetValue("FileProvider", out object? p)
                && p is FileConfigurationProvider configurationProvider
                    ? Path.GetDirectoryName(configurationProvider.Source.Path)
                    : Directory.GetCurrentDirectory();
        }
        else
        {
            directory = EnsureTrailingSlash(Path.GetFullPath(fileName));
        }

        Action<EnvironmentFileConfigurationSource> configureSource = s =>
        {
            s.Prefix = prefix;
            s.RemoveWrappingQuotes = removeWrappingQuotes;
            s.Trim = trim;
            s.Path = fileName;
            s.Optional = true;
            s.ReloadOnChange = reloadOnChange;
            s.FileProvider = new PhysicalFileProvider(directory!, ExclusionFilters.None);
        };
        return builder.Add(configureSource);
    }

    private static string EnsureTrailingSlash(string path)
    {
        return !string.IsNullOrEmpty(path) && path[path.Length - 1] != Path.DirectorySeparatorChar
            ? path + Path.DirectorySeparatorChar
            : path;
    }
}

These extensions make it easy to add our provider to the configuration pipeline with sensible defaults and customization options.

Integrating the Custom Provider

With all components in place, users can easily add this provider to their applications:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureAppConfiguration((hostingContext, config) =>
        {
            config
                .AddEnvironmentFile() // Use default .env file
                .AddEnvironmentFile("database.env") // Add another env file
                .AddEnvironmentVariables(); // Environment variables override file settings
        })
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });

Advanced Features

The EnvironmentFile provider demonstrates several advanced features you might want to consider for your own providers:

1. Reload on Change

The provider supports reloading configuration when the file changes:

config.AddEnvironmentFile("reloadable.env", reloadOnChange: true)

This is implemented by leveraging the FileConfigurationSource’s ReloadOnChange property, which triggers a file system watcher.

2. Prefix Handling

The provider can strip prefixes from keys:

config.AddEnvironmentFile("with-prefix.env", prefix: "MyPrefix_")

When using a prefix like MyPrefix_, keys like MyPrefix_MyVariable are loaded as just MyVariable.

3. Data Transformation

The provider handles specific transformations:

• Converting __ to : for hierarchical configuration
• Optionally removing quotes from values
• Optionally trimming whitespace

Best Practices for Custom Configuration Providers

Based on the EnvironmentFile provider, here are some best practices to consider:

1. Inherit from existing providers when possible to reuse functionality
2. Make your provider optional by default to avoid breaking the application if the configuration source is missing
3. Provide sensible defaults but allow customization through parameters
4. Use extension methods for a cleaner API
5. Add reload capabilities when it makes sense for your source
6. Implement proper error handling to gracefully handle malformed input
7. Separate parsing logic from the provider itself for better maintainability

When to Use (and Not Use) Custom Providers

Custom configuration providers are powerful, but they’re not always necessary. Consider these guidelines:

Use a custom provider when:

• You need to support a specific file format not covered by built-in providers
• You need to integrate with external configuration services
• You want to add specialized processing or validation to configuration values

Consider alternatives when:

• You can achieve your goal with existing providers and a bit of code
• Your source is only used in development, not in production
• You’re dealing with simple transformations that can be handled in startup code

Conclusion

Custom configuration providers in .NET offer a powerful way to extend the configuration system to meet specific needs. The Configuration.Extensions.EnvironmentFile package demonstrates how to implement a complete provider that integrates smoothly with the .NET configuration pipeline.

By understanding the architecture and implementation patterns shown in this example, you can create your own providers to handle specialized configuration needs in your applications.

Want to see more? Check out the full implementation of the Environment File Provider on GitHub or install the NuGet package directly.

Happy coding!