A string formatting library in 65 lines of C++

In this write-up, I will walk you through an implementation of a string formatting library for C++ I came up with for my video game. The end result came out really compact, at only 65 lines of code—providing a skeleton that can be supplemented with additional functionality at low cost.

Usage

Given a format buffer…

char buffer[64];
String_Buffer buf = {str, sizeof str};

…the fmt::format function provided by this library can be called with a format string parameter, containing the character sequence {} (a hole) where parameters are to be substituted, as well as the parameters themselves.

fmt::format(buf, "Hello, {}!", "world");
assert(strcmp(str, "Hello, world!") == 0);

When a literal {{ is needed, the { must be doubled—even when format arguments are not present.

fmt::format(buf, "Hello, {{}!");
assert(strcmp(str, "Hello, {}!") == 0);

Further, when a format argument is not present, no undefined behaviour occurs—the hole is rendered as the empty string.

fmt::format(buf, "empty {} hole");
assert(strcmp(str, "empty  hole") == 0);

Multiple format arguments can be specified as well.

fmt::format(buf, "[{}] [{}] {}", "main", "info", "Hewwo :3");
assert(strcmp(str, "[main] [info] Hewwo :3") == 0);

In case the buffer is not sufficiently large to contain the full string, the function writes as many characters as it can, and sets the String_Buffer’s len variable to the amount of characters required. That way, it is possible for the caller to tell if the buffer has been exhausted, and reallocate it to an appropriate size.

fmt::format(
	buf,
	"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
	"aaaaaaaaaaaaaaaaaa{} {}",
	"Vector", "Amber"
);
assert(strcmp(
	str,
	"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
	"aaaaaaaaaaaaaaaaaaV"
) == 0);
assert(buf.len == 74);

Additional functions can be written on top of this base functionality to improve ergonomics in real-world code. These are included in Ergonomic functions.

Problem statement

1. A string formatting library consists of a single function format. You give the function a format string, which describes the output shape, as well as a set of format parameters, which the function then substitutes into the format string, rendering them in a human-readable way.

2. The format function ought to write to a pre-allocated buffer of characters.

   This is a choice made in favour of simplicity: writing to a pre-allocated buffer can fail, but compared to arbitrary I/O, there is only one failure mode: the buffer is exhausted.

   Naturally, this cannot work in memory-constrained environments, such as embedded devices—where you would want to write to a small buffer and flush it in a loop to reduce memory usage—but this does not apply in the context of a desktop video game.

3. As already mentioned in the usage overview, if the buffer is full, the function should return the number of characters that would have been written, had the buffer capacity not been exceeded—such that the caller can choose to reallocate the backing buffer to an appropriate size, and try formatting again.

4. There has to be a format string.

   An example of a format string-less API is C++’s <iostream>. Instead of having a format string like printf, <iostream> opts to use overloads of operator<< to write to the output. This has the disadvantage of not being greppable (which is useful for debugging error logs), as well as not being localisable (because there is no format string that could be replaced at runtime).

   Additionally, I don’t want the format string to have extra specifiers such as C’s %d, %x, etc. specifying the type of output, or Python’s {:.3}, for specifying the style of output. The C approach is error-prone and inextensible, and the Python approach, while convenient, increases parser complexity and reduces greppability. Instead, the representation is defined only according to the formatted value’s type.

5. It has to have a small footprint.

   There exist plenty of string formatting libraries for C++, such as {fmt}, or even the recently introduced std::print, but they suffer from gigantic compile-time complexity through their heavy use of template metaprogramming.

   While my compilation time benchmark results for {fmt} weren’t as dire as those presented in their README, they still don’t paint a pretty picture—with a simple program using printf taking ~35 ms to compile, and the equivalent program using {fmt} taking ~200 ms.

   I also find the benefits of an open rather than closed API, as well as compile-time checked format strings, dubious. Instead, I want something lean and small, using basic features of the language, and easy enough to drop into your own project, then extend and modify according to your needs—in spirit of rxi’s simple serialisation system.

6. Simply using printf is not good enough.

Implementation walkthrough

We will start by defining the String_Buffer type, which also serves as the formatter’s state. It represents a user-provided string buffer with a capacity and a length.

struct String_Buffer
{
	char* str;
	int cap;
	int len = 0;
};

A String_Buffer is intended to be initialised via aggregate initialisation ({str, cap}.) This mimics the snprintf API, which accepts its buffer and size arguments in the same order.

At the core of the library’s output is write. It performs a bounds-checked write of a string with known length to the output string buffer.

void write(String_Buffer& buf, const char* str, int len)
{
	int remaining_cap = buf.cap - buf.len - 1; // leave one byte for NUL
	int write_len = len > remaining_cap ? remaining_cap : len;
	if (write_len > 0)
		memcpy(buf.str + buf.len, str, write_len);
	buf.len += len;
}

My implementation truncates the output if the buffer size is exhausted, but keeps incrementing the buffer’s len past cap, such that the caller can know the full number of characters written after all writes, and adjust accordingly. This is a deliberate choice coming from the fact that String_Buffer does not own the buffer’s allocation, and the fact that string formatting is a performance-sensitive piece of code, which will be called often in the game loop.

However, it is trivial to replace the length saturation logic with a call to realloc, should that be the more appropriate choice.

Having this base write function, we can implement a set of overloaded functions that will write out values of various types to the string buffer. These functions will be used by our format function, to write out format arguments.

The set of functions implemented here directly corresponds to the types of arguments you’ll be able to pass into format.

void write_value(String_Buffer& buf, const char* value)
{
    write(buf, value, strlen(value));
}

void write_value(String_Buffer& buf, bool value) { /* ... */ }
void write_value(String_Buffer& buf, char value) { /* ... */ }
void write_value(String_Buffer& buf, int value) { /* ... */ }

See write_value for various types for a set of example implementations for other types.

Now onto parsing format strings.

Format strings can be defined as a sequence of literals interspersed with arguments. That is, a format string always takes the form:

fstr = { literal, hole }, literal;

The leading and trailing literal can be the empty string.

The task of processing the literal parts is done by a function called next_hole. It parses the format string, looking for a character sequence representing a hole {}, and writes the string preceding the hole {} to the output buffer.

bool next_hole(String_Buffer& buf, const char*& fstr)
{
	const char* prefix = fstr;
	while (*fstr != 0) {
		if (*fstr == '{') {
			int len = fstr - prefix;
			++fstr;
			if (*fstr == '}') {
				++fstr;
				write(buf, prefix, len);
				return true;
			}
			if (*fstr == '{') {
				write(buf, prefix, len);
				prefix = fstr;
				++fstr;
			}
		}
		++fstr;
	}
	write(buf, prefix, fstr - prefix);
	return false;
}

fstr is received as a reference to a pointer, representing the format string’s parsing state.

A call to next_hole will write out the literal part, visualised with ---, and leave the fstr pointer past the hole {}, visualised with ^.

Hello, {}!
-------  ^

In this case, it will return true to signal that it stopped at a hole.
In case there is no hole however, and the end of the string is reached, it will return false.

Hello, {}!
         -^ end of string

Additionally, we handle the {{ escaping case. when { is encountered directly after another {, we have to flush the current literal, and start a new one directly after the first {. Underlined with --- are the spans of characters that get written to the output.

empty {{} hole
------- ------

Finally, we define format: the function that accepts a format string, a set of arguments, and inserts them into the output string. It makes use of an additional function format_value, which tries to find the next hole, and if found, writes out a format argument in its place.

template<typename T>
void format_value(String_Buffer& buf, const char* fstr, const T& value)
{
	if (next_hole(buf, fstr))
		write_value(buf, value);
}

template<typename... Args>
void format(String_Buffer& buf, const char* fstr, const Args&... args)
{
	(format_value(buf, fstr, args), ...);
	while (next_hole(buf, fstr)) {}
}

For those unfamiliar with C++ template metaprogramming, (format_value(buf, fstr, args), ...) is a fold expression. Given any number of args, it will expand into a sequence of calls to format_value, one for each element in args, separated by the , operator. For example, if two arguments: a const char* and an int, are passed into format:

template<>
void format<const char*, int>(
	String_Buffer& buf,
	const char* fstr,
	const char* a1, int a2)
{
	(format_value(buf, fstr, a1), format_value(buf, fstr, a2));
	while (next_hole(buf, fstr)) {}
}

Note that the overloads of write_value must be declared before format_value. This is because the write_value name is not dependent on any template arguments, and is therefore early-bound at format_value’s definition site.

This choice was made for the sake of simplicity, but if it turns out to be a problem, it is possible to use specialisation. It is important to note though that specialisation bypasses overload resolution, so this will not work:

template<typename T>
void write_value(String_Buffer& buf, T value) = delete;

template<>
void write_value<const char*>(
	String_Buffer& buf, const char* value)
{
	if (next_hole(buf, fstr))
		write(buf, value, strlen(value));
}

template<typename T>
void format_value(String_Buffer& buf, const char*& fstr, const T& value)
{
	if (next_hole(buf, fstr))
		write_value<T>(buf, value);
}

template<typename... Args>
void format(String_Buffer& buf, const char* fstr, const Args&... args)
{
	(format_value(buf, fstr, args), ...);
	while (next_hole(buf, fstr)) {}
}

format(buf, "Hello, {}!", "world");

because the type of "world" is char [5], and not const char*, and write_value<char [5]> is deleted. This should be solvable with some additional work, but I’ve deemed it unnecessary in my case.

In a single .cpp file, together with wrapping all the functionality in a namespace, this implementation, together with the implementation of write_value for strings, equates to a mere 65 lines of code.

In a real project, you will probably want to move some of the private implementation details to a separate .cpp file. Here’s the full source code listing, split into a header file, and an implementation file.

#pragma once

struct String_Buffer
{
	char* str;
	int cap;
	int len = 0;
};

namespace fmt {

void write_value(String_Buffer& buf, const char* value);
// (additional overloads here)

// implementation detail
bool next_hole(String_Buffer& buf, const char*& fstr);

template<typename T>
void format_value(String_Buffer& buf, const char*& fstr, const T& value)
{
	if (next_hole(buf, fstr))
		write_value(buf, value);
}

template<typename... Args>
void format(String_Buffer& buf, const char* fstr, const Args&... args)
{
	(format_value(buf, fstr, args), ...);
	while (next_hole(buf, fstr)) {}
}

}

#include "format.hpp"

#include <cstring>

namespace fmt
{

static void write(String_Buffer& buf, const char* str, int len)
{
	int remaining_cap = buf.cap - buf.len - 1; // leave one byte for NUL
	int write_len = len > remaining_cap ? remaining_cap : len;
	if (write_len > 0)
		memcpy(buf.str + buf.len, str, write_len);
	buf.len += len;
}

void write_value(String_Buffer& buf, const char*& fstr, const char* value)
{
	write(buf, value, strlen(value));
}

bool next_hole(String_Buffer& buf, const char*& fstr)
{
	const char* prefix = fstr;
	while (*fstr != 0) {
		if (*fstr == '{') {
			int len = fstr - prefix;
			++fstr;
			if (*fstr == '}') {
				++fstr;
				write(buf, prefix, len);
				return true;
			}
			if (*fstr == '{') {
				write(buf, prefix, len);
				prefix = fstr;
				++fstr;
			}
		}
		++fstr;
	}
	write(buf, prefix, fstr - prefix);
	return false;
}

}

Design remarks

Escaping ambiguity

The choice of {} as the hole syntax is not accidental. I evaluated whether holes could be represented with a single character %, like:

fmt::format(buf, "Hello, %!", "world");

But it turned that using only a single character introduces an ambiguity around escaping. What should this format to: hello%, or %hello?

fmt::format(buf, "%%%", "hello");

It would be possible to use a different, unambiguous combination for escaping, such as %_, but it looks very alien, and you have to use it any time you want a % sign.

fmt::format(buf, "%%_ complete", 33);

Compare this to the current approach, where you only have to double the { when it’s directly preceding }.

fmt::format(buf, "{}% complete", 33);

It also more closely mimics the final output string. Reading the previous %%_ example requires knowing that %_ is a special sequence that turns into %, whereas reading this example doesn’t require any extra knowledge (and progress reporting with percentages is a somewhat common use case for format strings).

Iteration through parameter packs

Another idea I had was to do an <iostream>-style API, though done with a function call rather than an operator chain:

format(buf, "Hello, ", "world!");

The observation about poor greppability didn’t occur to me until later, but it seemed simple enough to implement.

void format_value(String_Buffer& buf, const char* value);
void format_value(String_Buffer& buf, int value);

template<typename... Args>
void format(String_Buffer& buf, const Args&... args)
{
	(format_value(buf, args), ...);
}

If I went with this approach, it would be even less code, but the poor greppability and non-localisability of format strings kept bugging me, so I stared wondering if there’s some way to add that format string.

It seemed impossible, because the format string can be provided at runtime. This would mean format would have to iterate through the format string to parse out the holes {}, and when a hole is hit, insert the Nth parameter, starting with 0 for the first hole, N for the last hole. But it seemed to require indexing the parameter pack, and

• there is no way to index a parameter pack in C++20,
• there is no way to index it using a runtime value in C++26, which adds parameter pack indexing pack...[x].

A few hours later, I realised it is possible to have the parameter pack expansion drive the parsing, rather than driving the parsing from format and trying to index the parameter pack.

I think this is single-handedly the most elegant bit of this library.

• It generates optimal, extremely minimal code: a sequence of calls to the appropriate overloads of format_value.
• It handles out-of-bounds gracefully: because there is no indexing of parameters, and therefore no out-of-bounds.

It makes me wonder what other cool things could be done with this technique.

Failed idea: using dynamic typing for format arguments

My initial idea for a minimal C++ formatting library involved a Format_Argument type, passed in an std::initializer_list to the format function. The API was shaped like this:

enum class Format_Argument_Type
{
	boolean,
	int32,
	float32,
	vec4,
};

struct Format_Argument
{
	Format_Argument_Type type;
	union
	{
		bool b;
		int32_t i;
		float f;
		Vec4 v;
	};
};

void format_value(String_Buffer& buf, const Format_Argument& arg);
void format(
	String_Buffer& buf,
	const char* fstr,
	std::initializer_list<Format_Argument> args);

This approach has a couple problems though, which were enough of a deal breaker for me that I dropped the idea.

• Efficiency.

  The size of Format_Argument is as large as the biggest value able to be formatted. In this case, assuming Vec4 is four 32-bit floats, it is 20 bytes. This space has to be allocated on the stack for the initializer_list.

  It is unlikely compilers would be able to optimise all that away, especially if the format function lived in a separate object file.

• Verbosity.

  The example above is actually incomplete. What Format_Argument has to look like is actually this:

  struct Format_Argument
  {
  	Format_Argument_Type type;
  	union
  	{
  		bool b;
  		int32_t i;
  		float f;
  		Vec4 v;
  	};
  
  	Format_Argument(bool b) : type(Format_Argument_Type::boolean), b(b) {}
  	Format_Argument(int32_t i) : type(Format_Argument_Type::int32), i(i) {}
  	Format_Argument(float f) : type(Format_Argument_Type::float32), f(f) {}
  	Format_Argument(Vec4 v) : type(Format_Argument_Type::vec4), v(v) {}
  };
  

  And then you have to switch on the format argument’s type in format_value, introducing further duplication.

Why not printf

The elephant in the room. Why do this when you have printf?

The answer to this is: verbosity.

Firstly, there is no way to extend printf with your own types in standard C. I often want to printf 3D vectors for debugging, and I have to resort to listing out all the axes manually.

printf(
	"%f %f %f",
	player.position.x,
	player.position.y,
	player.position.z
);

I think you can see how this gets old real quick.

Combine this with the inability to use printf as an expression, which is particularly painful with ImGui—where I often want to format a window title, or button label.

char entity_name[64];
snprintf(
	entity_name, sizeof entity_name,
	"%d(%d) %s",
	entity_id.index, entity_id.generation,
	entity_kind::names[entity.kind]
);
if (ImGui::TreeNode(entity_name)) {
	// ...
	ImGui::TreePop();
}

It is possible to write a function which allocates the temporary buffer and writes to it in one go, akin to my fmt::print function, but even doing that is verbose, as you have to deal with va_list—therefore needing two sets of functions, one for variadic arguments ... and one for va_list.

printf is also error-prone. It is easy to mess up and use the wrong specifier type, or pass too few arguments to the function.

printf("%x", 1.0f); // oops
printf("%x");       // ...not again

This makes it unusable for localisation purposes.

There is also no easy, idiomatic way to concatenate strings written with snprintf.

char str[4] = {0};
int cursor = 0;
cursor += snprintf(str, sizeof str, "hello ");
cursor += snprintf(str, sizeof str, "world!");

This naive way is not actually correct, because snprintf returns the number of characters that would be written into str, had the buffer been large enough. Therefore, the second call to snprintf in the above example ends up writing past the buffer’s bounds (at index 6.)

Extras

Since the base library is very bare-bones, I’m including some additional snippets to help you get it integrated into your project.

write_value for various types

void write_value(String_Buffer& buf, const char* value)
{
	write(buf, value, int(strlen(value)));
}

void write_value(String_Buffer& buf, bool value)
{
	if (value)
		write(buf, "true", 4);
	else
		write(buf, "false", 5);
}

void write_value(String_Buffer& buf, char value)
{
	write(buf, &value, 1);
}

For integers, here’s an implementation of write_value for int64_t. This can confuse C++’s overload resolution, so I’d recommend adding additional overloads for smaller integers int8_t, int16_t, int32_t, also long long, and ptrdiff_t, calling into the int64_t overload.

void write_value(String_Buffer& buf, int64_t value)
{
	if (value == 0) {
		write(buf, "0", 1);
		return;
	}
	if (value < 0) {
		write(buf, "-", 1);
		value = -value;
	}

	char digits[20] = {};
	int i = sizeof digits - 1;
	while (value > 0) {
		digits[i--] = '0' + (value % 10);
		value /= 10;
	}
	int ndigits = sizeof digits - i - 1;
	write(buf, digits + i + 1, ndigits);
}

A uint64_t version can be created in a similar manner, by removing the if (value < 0) case near the beginning.

This algorithm works for any radix (base 2, base 8, base 16, …). In my own implementation, I have a Format_Hex newtype, which changes the output to base 16.

struct Format_Hex
{
	uint64_t value;
};

namespace fmt
{

inline Format_Hex hex(uint64_t value) { return {value}; }

}

For floats, I defer the work onto snprintf’s %g specifier, because I trust it to do a better job than I ever could, even if a bit slow. You can also use Ryu for this purpose.

void write_value(String_Buffer& buf, double value)
{
	char f[32] = {};
	int len = snprintf(f, sizeof f, "%g", value);
	if (len > sizeof f - 1)
		len = sizeof f - 1;
	write(buf, f, len);
}

And of course, don’t forget about vectors—which were one of my motivating examples for abandoning printf.

void write_value(String_Buffer& buf, Vec3 value)
{
	write(buf, "(", 1);
	write_value(buf, value.x);
	write(buf, ", ", 2);
	write_value(buf, value.y);
	write(buf, ", ", 2);
	write_value(buf, value.z);
	write(buf, ")", 1);
}

Ergonomic functions

The ergonomics of having to allocate a backing buffer, and then a String_Buffer afterwards, can get a bit cumbersome. To help alleviate this, I have a Static_String type, together with a print function, which formats to a Static_String and returns it:

template<int N>
struct Static_String
{
	char data[N] = {};

	const char* operator*() const { return data; }
};

namespace fmt
{

template<int N, typename... Args>
Static_String<N> print(const char* fstr, const Args&... args)
{
	Static_String<N> str;
	String_Buffer buf = {str.data, sizeof str.data};
	format(buf, fstr, args...);
	return str;
}

}

This makes it very easy to use a format string wherever an ordinary const char* is expected.

if (ImGui::TreeNode(
	*fmt::print<64>("{}({}) {}", index, generation, entity_kind)
)) {
	// ...
	ImGui::TreePop();
}

Thank you to my friend Tori for giving a whole bunch of solid feedback on a draft of this post.