Functions related to low-level strings. More...

#include <iosfwd>
#include "core/bitmath_func.hpp"
#include "string_type.h"

Data Structures
struct	CaseInsensitiveComparator
	Case insensitive comparator for strings, for example for use in std::map. More...

Functions
void	strecpy (std::span< char > dst, std::string_view src)
	Copies characters from one buffer to another. More...

std::string	FormatArrayAsHex (std::span< const uint8_t > data)
	Format a byte array into a continuous hex string. More...

void	StrMakeValidInPlace (char str, const char last, StringValidationSettings settings=SVS_REPLACE_WITH_QUESTION_MARK) NOACCESS(2)
	Scans the string for invalid characters and replaces then with a question mark '?' (if not ignored). More...

std::string	StrMakeValid (std::string_view str, StringValidationSettings settings=SVS_REPLACE_WITH_QUESTION_MARK)
	Copies the valid (UTF-8) characters from `str` to the returned string. More...

void	StrMakeValidInPlace (char *str, StringValidationSettings settings=SVS_REPLACE_WITH_QUESTION_MARK)
	Scans the string for invalid characters and replaces then with a question mark '?' (if not ignored). More...

bool	strtolower (std::string &str, std::string::size_type offs=0)

bool	StrValid (std::span< const char > str)
	Checks whether the given string is valid, i.e. More...

void	StrTrimInPlace (std::string &str)
	Trim the spaces from given string in place, i.e. More...

std::string_view	StrTrimView (std::string_view str)

bool	StrStartsWithIgnoreCase (std::string_view str, const std::string_view prefix)
	Check whether the given string starts with the given prefix, ignoring case. More...

bool	StrEndsWithIgnoreCase (std::string_view str, const std::string_view suffix)
	Check whether the given string ends with the given suffix, ignoring case. More...

int	StrCompareIgnoreCase (const std::string_view str1, const std::string_view str2)
	Compares two string( view)s, while ignoring the case of the characters. More...

bool	StrEqualsIgnoreCase (const std::string_view str1, const std::string_view str2)
	Compares two string( view)s for equality, while ignoring the case of the characters. More...

int	StrNaturalCompare (std::string_view s1, std::string_view s2, bool ignore_garbage_at_front=false)
	Compares two strings using case insensitive natural sort. More...

bool	StrNaturalContains (const std::string_view str, const std::string_view value)
	Checks if a string is contained in another string with a locale-aware comparison that is case sensitive. More...

bool	StrNaturalContainsIgnoreCase (const std::string_view str, const std::string_view value)
	Checks if a string is contained in another string with a locale-aware comparison that is case insensitive. More...

bool	ConvertHexToBytes (std::string_view hex, std::span< uint8_t > bytes)
	Convert a hex-string to a byte-array, while validating it was actually hex. More...

bool	StrEmpty (const char *s)
	Check if a string buffer is empty. More...

size_t	ttd_strnlen (const char *str, size_t maxlen)
	Get the length of a string, within a limited buffer. More...

bool	IsValidChar (char32_t key, CharSetFilter afilter)
	Only allow certain keys. More...

size_t	Utf8Decode (char32_t c, const char s)
	Decode and consume the next UTF-8 encoded character. More...

size_t	Utf8Encode (char *buf, char32_t c)

size_t	Utf8Encode (std::ostreambuf_iterator< char > &buf, char32_t c)

size_t	Utf8Encode (std::back_insert_iterator< std::string > &buf, char32_t c)

size_t	Utf8TrimString (char *s, size_t maxlen)
	Properly terminate an UTF8 string to some maximum length. More...

char32_t	Utf8Consume (const char **s)

template<class Titr >
char32_t	Utf8Consume (Titr &s)

int8_t	Utf8CharLen (char32_t c)
	Return the length of a UTF-8 encoded character. More...

int8_t	Utf8EncodedCharLen (char c)
	Return the length of an UTF-8 encoded value based on a single char. More...

bool	IsUtf8Part (char c)

char *	Utf8PrevChar (char *s)
	Retrieve the previous UNICODE character in an UTF-8 encoded string. More...

const char *	Utf8PrevChar (const char *s)

size_t	Utf8StringLength (const char *s)
	Get the length of an UTF-8 encoded string in number of characters and thus not the number of bytes that the encoded string contains. More...

size_t	Utf8StringLength (const std::string &str)
	Get the length of an UTF-8 encoded string in number of characters and thus not the number of bytes that the encoded string contains. More...

bool	Utf16IsLeadSurrogate (uint c)
	Is the given character a lead surrogate code point? More...

bool	Utf16IsTrailSurrogate (uint c)
	Is the given character a lead surrogate code point? More...

char32_t	Utf16DecodeSurrogate (uint lead, uint trail)
	Convert an UTF-16 surrogate pair to the corresponding Unicode character. More...

char32_t	Utf16DecodeChar (const uint16_t *c)
	Decode an UTF-16 character. More...

bool	IsTextDirectionChar (char32_t c)
	Is the given character a text direction character. More...

bool	IsPrintable (char32_t c)

bool	IsWhitespace (char32_t c)
	Check whether UNICODE character is whitespace or not, i.e. More...

Detailed Description

Functions related to low-level strings.

Definition in file string_func.h.

Function Documentation

◆ ConvertHexToBytes()

bool ConvertHexToBytes	(	std::string_view	hex,
		std::span< uint8_t >	bytes
	)

Convert a hex-string to a byte-array, while validating it was actually hex.

Parameters

hex	The hex-string to convert.
bytes	The byte-array to write the result to.

Note: The length of the hex-string has to be exactly twice that of the length of the byte-array, otherwise conversion will fail.

Returns: True iff the hex-string was valid and the conversion succeeded.

Definition at line 734 of file string.cpp.

References ConvertHexNibbleToByte().

Referenced by X25519AuthorizedKeyClientHandler::GetValidSecretKeyAndUpdatePublicKey(), GRFLoadConfig(), PickerLoadConfig(), and ValidateSignature().

◆ FormatArrayAsHex()

std::string FormatArrayAsHex ( std::span< const uint8_t > data )

Format a byte array into a continuous hex string.

Parameters

data	Array to format

Returns: Converted string.

Definition at line 81 of file string.cpp.

Referenced by AddGrfInfo(), CalculateHashV1(), GenerateUid(), X25519AuthenticationHandler::GetPeerPublicKey(), X25519AuthorizedKeyClientHandler::GetValidSecretKeyAndUpdatePublicKey(), HandleSavegameLoadCrash(), IsGoodGRFConfigList(), CommandHelperBase::LogCommandExecution(), NetworkContentListWindow::OpenExternalSearch(), OutputContentState(), ClientNetworkGameSocketHandler::Receive_SERVER_CHECK_NEWGRFS(), and SurveyGrfs().

◆ IsTextDirectionChar()

bool IsTextDirectionChar ( char32_t c )

inline

Is the given character a text direction character.

Parameters

c	The character to test.

Returns: true iff the character is used to influence the text direction.

Definition at line 217 of file string_func.h.

References CHAR_TD_LRE, CHAR_TD_LRM, CHAR_TD_LRO, CHAR_TD_PDF, CHAR_TD_RLE, CHAR_TD_RLM, and CHAR_TD_RLO.

◆ IsValidChar()

bool IsValidChar	(	char32_t	key,
		CharSetFilter	afilter
	)

Only allow certain keys.

You can define the filter to be used. This makes sure no invalid keys can get into an editbox, like BELL.

Parameters

key	character to be checked
afilter	the filter to use

Returns: true or false depending if the character is printable/valid or not

Definition at line 396 of file string.cpp.

References CS_ALPHANUMERAL.

Referenced by IConsoleCmdExec().

◆ IsWhitespace()

bool IsWhitespace ( char32_t c )

inline

Check whether UNICODE character is whitespace or not, i.e.

whether this is a potential line-break character.

Parameters

c	UNICODE character to check

Returns: a boolean value whether 'c' is a whitespace character or not

See also: http://www.fileformat.info/info/unicode/category/Zs/list.htm

Definition at line 249 of file string_func.h.

Referenced by IConsoleHistoryAdd(), IcuStringIterator::Next(), FallbackParagraphLayout::NextLine(), and IcuStringIterator::Prev().

◆ StrCompareIgnoreCase()

int StrCompareIgnoreCase	(	const std::string_view	str1,
		const std::string_view	str2
	)

Compares two string( view)s, while ignoring the case of the characters.

Parameters

str1	The first string.
str2	The second string.

Returns: Less than zero if str1 < str2, zero if str1 == str2, greater than zero if str1 > str2. All ignoring the case of the characters.

Definition at line 334 of file string.cpp.

Referenced by MatchesExtension(), and StrEqualsIgnoreCase().

◆ strecpy()

void strecpy	(	std::span< char >	dst,
		std::string_view	src
	)

Copies characters from one buffer to another.

Copies the source string to the destination buffer with respect of the terminating null-character and the size of the destination buffer.

Note: usage: strecpy(dst, src);

Parameters

dst	The destination buffer
src	The buffer containing the string to copy

Definition at line 60 of file string.cpp.

Referenced by FileStringReader::HandlePragma().

◆ StrEmpty()

bool StrEmpty ( const char * s )

inline

Check if a string buffer is empty.

Parameters

s	The pointer to the first element of the buffer

Returns: true if the buffer starts with the terminating null-character or if the given pointer points to nullptr else return false

Definition at line 57 of file string_func.h.

Referenced by GetEnv(), GRFConfig::GetName(), IConsoleCmdExec(), IConsoleHistoryAdd(), FileStringReader::ParseFile(), RenameSign(), and MusicDriver_ExtMidi::Start().

◆ StrEndsWithIgnoreCase()

bool StrEndsWithIgnoreCase	(	std::string_view	str,
		const std::string_view	suffix
	)

Check whether the given string ends with the given suffix, ignoring case.

Parameters

str	The string to look at.
suffix	The suffix to look for.

Returns: True iff the end of the string is the same as the suffix, ignoring case.

Definition at line 321 of file string.cpp.

References StrEqualsIgnoreCase().

Referenced by TextfileWindow::AfterLoadText().

◆ StrEqualsIgnoreCase()

bool StrEqualsIgnoreCase	(	const std::string_view	str1,
		const std::string_view	str2
	)

Compares two string( view)s for equality, while ignoring the case of the characters.

Parameters

str1	The first string.
str2	The second string.

Returns: True iff both strings are equal, barring the case of the characters.

Definition at line 347 of file string.cpp.

References StrCompareIgnoreCase().

Referenced by ChangeWorkingDirectoryToExecutable(), DEF_CONSOLE_CMD(), AIScannerInfo::FindInfo(), GameScannerInfo::FindInfo(), FiosGetSavegameListCallback(), FiosGetScenarioListCallback(), FiosGetTownDataListCallback(), BlitterFactory::GetBlitterFactory(), ScriptScanner::RegisterScript(), StrEndsWithIgnoreCase(), StringToContentType(), and StrStartsWithIgnoreCase().

◆ StrMakeValid()

std::string StrMakeValid	(	std::string_view	str,
		StringValidationSettings	settings
	)

Copies the valid (UTF-8) characters from str to the returned string.

Depending on the settings invalid characters can be replaced with a question mark, as well as determining what characters are deemed invalid.

Parameters

str	The string to validate.
settings	The settings for the string validation.

Definition at line 205 of file string.cpp.

References settings, and StrMakeValid().

◆ StrMakeValidInPlace() [1/2]

void StrMakeValidInPlace	(	char *	str,
		const char *	last,
		StringValidationSettings	settings
	)

Scans the string for invalid characters and replaces then with a question mark '?' (if not ignored).

Parameters

str	The string to validate.
last	The last valid character of str.
settings	The settings for the string validation.

Definition at line 178 of file string.cpp.

References settings, and StrMakeValid().

Referenced by Textbuf::Assign(), and StrMakeValidInPlace().

◆ StrMakeValidInPlace() [2/2]

void StrMakeValidInPlace	(	char *	str,
		StringValidationSettings	settings
	)

Scans the string for invalid characters and replaces then with a question mark '?' (if not ignored).

Only use this function when you are sure the string ends with a '\0'; otherwise use StrMakeValidInPlace(str, last, settings) variant.

Parameters

str	The string (of which you are sure ends with '\0') to validate.

Definition at line 192 of file string.cpp.

References settings, and StrMakeValidInPlace().

◆ StrNaturalCompare()

int StrNaturalCompare	(	std::string_view	s1,
		std::string_view	s2,
		bool	ignore_garbage_at_front
	)

Compares two strings using case insensitive natural sort.

Parameters

s1	First string to compare.
s2	Second string to compare.
ignore_garbage_at_front	Skip punctuation characters in the front

Returns: Less than zero if s1 < s2, zero if s1 == s2, greater than zero if s1 > s2.

Definition at line 589 of file string.cpp.

References _current_collator, and SkipGarbage().

Referenced by CargoSpecNameSorter(), GRFSorter(), IndustryDirectoryWindow::IndustryNameSorter(), IndustryTypeNameSorter(), NetworkContentListWindow::NameSorter(), NewGRFWindow::NameSorter(), DropDownString< TBase, TFs, TEnd >::NatSortFunc(), NetworkGameWindow::NGameNameSorter(), SignList::SignNameSorter(), CompanyStationsWindow::StationNameSorter(), TownDirectoryWindow::TownNameSorter(), and NetworkContentListWindow::TypeSorter().

◆ StrNaturalContains()

bool StrNaturalContains	(	const std::string_view	str,
		const std::string_view	value
	)

Checks if a string is contained in another string with a locale-aware comparison that is case sensitive.

Parameters

str	The string to search in.
value	The string to search for.

Returns: True if a match was found.

Definition at line 660 of file string.cpp.

References ICUStringContains(), MacOSStringContains(), and Win32StringContains().

Referenced by StringFilter::AddLine().

◆ StrNaturalContainsIgnoreCase()

bool StrNaturalContainsIgnoreCase	(	const std::string_view	str,
		const std::string_view	value
	)

Checks if a string is contained in another string with a locale-aware comparison that is case insensitive.

Parameters

str	The string to search in.
value	The string to search for.

Returns: True if a match was found.

Definition at line 687 of file string.cpp.

References ICUStringContains(), MacOSStringContains(), and Win32StringContains().

Referenced by StringFilter::AddLine().

◆ StrStartsWithIgnoreCase()

bool StrStartsWithIgnoreCase	(	std::string_view	str,
		const std::string_view	prefix
	)

Check whether the given string starts with the given prefix, ignoring case.

Parameters

str	The string to look at.
prefix	The prefix to look for.

Returns: True iff the begin of the string is the same as the prefix, ignoring case.

Definition at line 281 of file string.cpp.

References StrEqualsIgnoreCase().

◆ StrTrimInPlace()

void StrTrimInPlace ( std::string & str )

Trim the spaces from given string in place, i.e.

the string buffer that is passed will be modified whenever spaces exist in the given string. When there are spaces at the begin, the whole string is moved forward and when there are spaces at the back the '\0' termination is moved.

Parameters

str	The string to perform the in place trimming on.

Definition at line 260 of file string.cpp.

Referenced by CheckClientAndServerName(), NetworkValidateClientName(), and NetworkValidateServerName().

◆ StrValid()

bool StrValid ( std::span< const char > str )

Checks whether the given string is valid, i.e.

contains only valid (printable) characters and is properly terminated.

Note: std::span is used instead of std::string_view as we are validating fixed-length string buffers, and std::string_view's constructor will assume a C-string that ends with a NUL terminator, which is one of the things we are checking.

Parameters

str	Span of chars to validate.

Definition at line 227 of file string.cpp.

References Utf8Decode(), and Utf8EncodedCharLen().

◆ ttd_strnlen()

size_t ttd_strnlen	(	const char *	str,
		size_t	maxlen
	)

inline

Get the length of a string, within a limited buffer.

Parameters

str	The pointer to the first element of the buffer
maxlen	The maximum size of the buffer

Returns: The length of the string

Definition at line 69 of file string_func.h.

◆ Utf16DecodeChar()

char32_t Utf16DecodeChar ( const uint16_t * c )

inline

Decode an UTF-16 character.

Parameters

c	Pointer to one or two UTF-16 code points.

Returns: Decoded Unicode character.

Definition at line 202 of file string_func.h.

Referenced by IcuStringIterator::Next(), and IcuStringIterator::Prev().

◆ Utf16DecodeSurrogate()

char32_t Utf16DecodeSurrogate	(	uint	lead,
		uint	trail
	)

inline

Convert an UTF-16 surrogate pair to the corresponding Unicode character.

Parameters

lead	Lead surrogate code point.
trail	Trail surrogate code point.

Returns: Decoded Unicode character.

Definition at line 192 of file string_func.h.

Referenced by HandleCharMsg().

◆ Utf16IsLeadSurrogate()

bool Utf16IsLeadSurrogate ( uint c )

inline

Is the given character a lead surrogate code point?

Parameters

c	The character to test.

Returns: True if the character is a lead surrogate code point.

Definition at line 171 of file string_func.h.

Referenced by HandleCharMsg().

◆ Utf16IsTrailSurrogate()

bool Utf16IsTrailSurrogate ( uint c )

inline

Is the given character a lead surrogate code point?

Parameters

c	The character to test.

Returns: True if the character is a lead surrogate code point.

Definition at line 181 of file string_func.h.

Referenced by HandleCharMsg().

◆ Utf8CharLen()

int8_t Utf8CharLen ( char32_t c )

inline

Return the length of a UTF-8 encoded character.

Parameters

c	Unicode character.

Returns: Length of UTF-8 encoding for character.

Definition at line 105 of file string_func.h.

◆ Utf8Decode()

size_t Utf8Decode	(	char32_t *	c,
		const char *	s
	)

Decode and consume the next UTF-8 encoded character.

Parameters

c	Buffer to place decoded character.
s	Character stream to retrieve character from.

Returns: Number of characters in the sequence.

Definition at line 419 of file string.cpp.

References GB(), and HasBit().

Referenced by FixSCCEncoded(), SkipGarbage(), StrValid(), and TranslateTTDPatchCodes().

◆ Utf8EncodedCharLen()

int8_t Utf8EncodedCharLen ( char c )

inline

Return the length of an UTF-8 encoded value based on a single char.

This char should be the first byte of the UTF-8 encoding. If not, or encoding is invalid, return value is 0

Parameters

c	char to query length of

Returns: requested size

Definition at line 124 of file string_func.h.

References GB().

Referenced by FixSCCEncoded(), StrValid(), TranslateTTDPatchCodes(), and Utf8TrimString().

◆ Utf8PrevChar()

char* Utf8PrevChar ( char * s )

inline

Retrieve the previous UNICODE character in an UTF-8 encoded string.

Parameters

s	char pointer pointing to (the first char of) the next character

Returns: a pointer in 's' to the previous UNICODE character's first byte

Note: The function should not be used to determine the length of the previous encoded char because it might be an invalid/corrupt start-sequence

Definition at line 149 of file string_func.h.

Referenced by Textbuf::Assign().

◆ Utf8StringLength() [1/2]

size_t Utf8StringLength ( const char * s )

Get the length of an UTF-8 encoded string in number of characters and thus not the number of bytes that the encoded string contains.

Parameters

s	The string to get the length for.

Returns: The length of the string in characters.

Definition at line 359 of file string.cpp.

Referenced by Textbuf::Assign(), CmdAlterGroup(), CmdPlaceSign(), CmdRenameCompany(), CmdRenameDepot(), CmdRenameEngine(), CmdRenamePresident(), CmdRenameSign(), CmdRenameStation(), CmdRenameTown(), CmdRenameVehicle(), CmdRenameWaypoint(), Utf8StringLength(), and VerifyTownName().

◆ Utf8StringLength() [2/2]

size_t Utf8StringLength ( const std::string & str )

Get the length of an UTF-8 encoded string in number of characters and thus not the number of bytes that the encoded string contains.

Parameters

s	The string to get the length for.

Returns: The length of the string in characters.

Definition at line 373 of file string.cpp.

References Utf8StringLength().

◆ Utf8TrimString()

size_t Utf8TrimString	(	char *	s,
		size_t	maxlen
	)

Properly terminate an UTF8 string to some maximum length.

Parameters

s	string to check if it needs additional trimming
maxlen	the maximum length the buffer can have.

Returns: the new length in bytes of the string (eg. strlen(new_string))

Note: maxlen is the string length INCLUDING the terminating '\0'

Definition at line 508 of file string.cpp.

References Utf8EncodedCharLen().

Data Structures

Functions

Detailed Description

Function Documentation

◆ ConvertHexToBytes()

◆ FormatArrayAsHex()

◆ IsTextDirectionChar()

◆ IsValidChar()

◆ IsWhitespace()

◆ StrCompareIgnoreCase()

◆ strecpy()

◆ StrEmpty()

◆ StrEndsWithIgnoreCase()

◆ StrEqualsIgnoreCase()

◆ StrMakeValid()

◆ StrMakeValidInPlace() [1/2]

◆ StrMakeValidInPlace() [2/2]

◆ StrNaturalCompare()

◆ StrNaturalContains()

◆ StrNaturalContainsIgnoreCase()

◆ StrStartsWithIgnoreCase()

◆ StrTrimInPlace()

◆ StrValid()

◆ ttd_strnlen()

◆ Utf16DecodeChar()

◆ Utf16DecodeSurrogate()

◆ Utf16IsLeadSurrogate()

◆ Utf16IsTrailSurrogate()

◆ Utf8CharLen()

◆ Utf8Decode()

◆ Utf8EncodedCharLen()

◆ Utf8PrevChar()

◆ Utf8StringLength() [1/2]

◆ Utf8StringLength() [2/2]

◆ Utf8TrimString()