Add DECIMAL Type Support to C# Language Extension

[monamaki]

Summary

This PR implements full support for SQL Server DECIMAL type in the C# language extension, enabling seamless conversion between SQL Server's 19-byte SQL_NUMERIC_STRUCT format and .NET's SqlDecimal type. The implementation introduces a new SqlNumericHelper utility class and simplifies existing code by leveraging SqlDecimal's built-in capabilities.

Why These Changes?

Problem: The C# language extension lacked proper support for SQL Server's DECIMAL types, which are critical for financial, scientific, and precision-sensitive applications. Without this support:

• Stored procedures and user-defined functions using DECIMAL parameters couldn't be called from C# extensions
• Result sets containing DECIMAL columns couldn't be properly processed
• OUTPUT parameters of type DECIMAL returned incorrect or corrupted values

Solution: Implement bidirectional conversion between SQL Server's native SQL_NUMERIC_STRUCT (ODBC 19-byte format) and .NET's SqlDecimal type, with proper handling of:

• Precision (1-38 digits) and scale (0-precision)
• NULL values using SqlDecimal.Null
• OUTPUT parameter sentinel detection (uninitialized structs with precision=0 used as a sentinel/marker value to indicate OUTPUT parameters)
• Signed values (positive/negative decimals)

What Changed?

1. SqlNumericHelper.cs

Created a comprehensive utility class for DECIMAL conversions with five core methods:

• ToSqlDecimal(SqlNumericStruct): Converts SQL 19-byte struct → SqlDecimal

  • Handles sign bit, precision, scale validation
  • Reconstructs 128-bit value from four 32-bit integers

• FromSqlDecimal(SqlDecimal, precision, scale): Converts SqlDecimal to SQL struct

  • Adjusts scale using SqlDecimal.AdjustScale() when needed
  • Uses SqlDecimal.Precision property (auto-updated by framework)
  • Validates precision bounds (1-38), throws for overflow
  • Properly handles NULL values (creates zero-initialized struct)

• ToSqlDecimalFromPointer(SqlNumericStruct*): Unsafe pointer version for OUTPUT parameters

  • Detects OUTPUT parameter sentinel (precision=0 → returns SqlDecimal.Null)
  • Centralizes OUTPUT parameter convention handling
  • Prevents duplicate sentinel detection logic across codebase

• ToSqlNumericStructPointer(SqlDecimal, precision, scale): Pins managed memory for native interop

  • Boxes struct into array for heap allocation (avoids stack destruction)
  • Returns GCHandle to prevent garbage collection during native access

• GetSqlNumericStructPointer(GCHandle): Extracts pinned pointer from GCHandle

2. CSharpDecimalTests.cpp

Added comprehensive test coverage with 8 new decimal-specific tests:

Test Name	Purpose	Coverage
GetDecimalOutputParamTest	OUTPUT parameter handling	Sentinel detection, proper value assignment
DecimalPrecisionScaleTest	Precision/scale validation	All valid combinations (1-38 precision, 0-precision scale)
DecimalBoundaryValuesTest	Edge cases	Min/max values, zero, near-overflow scenarios
DecimalStructLayoutTest	Memory layout verification	19-byte ODBC struct correctness
GetDecimalInputColumnsTest	Input column processing	Batch decimal column data handling
GetDecimalResultColumnsTest	Output column generation	Result set decimal columns
DecimalColumnsWithNullsTest	NULL handling	Mixed NULL/non-NULL decimal columns
DecimalHighScaleTest	High precision scenarios	38-digit precision

Test Infrastructure Updates

3. CSharpTestExecutor.cs

Added managed test execution helper for decimal scenarios:

• ExecuteDecimalInputOutput: Tests input columns + OUTPUT parameters
• ExecuteDecimalResultSet: Tests decimal return columns

4. CSharpExtensionApiTests.h/cpp

Extended C++ test framework with decimal test declarations and utility functions.

5. CSharpInitParamTests.cpp

Added InitNumericParamTest for parameter initialization validation.

6. CSharpExecuteTests.cpp

Integrated decimal tests into main test suite execution.

10. Microsoft.SqlServer.CSharpExtension.csproj

Added SqlNumericHelper.cs to build configuration.

11. Microsoft.SqlServer.CSharpExtensionTest.csproj

Added CSharpTestExecutor.cs to test project.

Checklist

• Code builds successfully (Release configuration)
• All unit tests passing (68/68)
• New tests added for decimal support (8 comprehensive tests)
• Code follows existing coding standards
• Comments are concise and explain "why" not "what"

---

[Copilot]

Pull request overview

This PR adds end-to-end SQL Server DECIMAL/NUMERIC support to the .NET Core C# language extension by introducing an ODBC-compatible SQL_NUMERIC_STRUCT representation in managed code and wiring conversions to/from SqlDecimal, plus expanding the native/managed test coverage for decimal parameters and columns.

Changes:

• Introduces SqlNumericHelper with SQL_NUMERIC_STRUCT layout and conversion helpers to/from SqlDecimal.
• Updates parameter and dataset marshalling to support SqlDecimal/SQL_C_NUMERIC for input columns, output columns, and OUTPUT parameters.
• Adds new native and managed tests and updates test harness templates/instantiations for SQL_NUMERIC_STRUCT.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
language-extensions/dotnet-core-CSharp/test/src/native/CSharpInitParamTests.cpp	Adds InitParam template specialization for SQL_NUMERIC_STRUCT to pass precision/scale to InitParam.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpExtensionApiTests.cpp	Adds InitializeColumns specialization for numeric columns to use precision rather than sizeof.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpExecuteTests.cpp	Adds explicit template instantiation for executing numeric column tests.
language-extensions/dotnet-core-CSharp/test/src/native/CSharpDecimalTests.cpp	Adds a new native test suite covering decimal params, output params, and decimal columns.
language-extensions/dotnet-core-CSharp/test/src/managed/Microsoft.SqlServer.CSharpExtensionTest.csproj	Updates build output paths and adds Microsoft.Data.SqlClient dependency for managed tests.
language-extensions/dotnet-core-CSharp/test/src/managed/CSharpTestExecutor.cs	Adds managed executors to drive decimal OUTPUT parameter and precision-overflow scenarios.
language-extensions/dotnet-core-CSharp/test/include/CSharpExtensionApiTests.h	Adds max precision constant and a helper to build SQL_NUMERIC_STRUCT test values.
language-extensions/dotnet-core-CSharp/src/managed/utils/SqlNumericHelper.cs	New utility for SQL_NUMERIC_STRUCT layout plus conversion logic to/from SqlDecimal.
language-extensions/dotnet-core-CSharp/src/managed/utils/Sql.cs	Adds mapping and size metadata for NUMERIC/DECIMAL (SqlDecimal + SQL_NUMERIC_STRUCT).
language-extensions/dotnet-core-CSharp/src/managed/Microsoft.SqlServer.CSharpExtension.csproj	Adjusts output path defaults and adds Microsoft.Data.SqlClient package reference.
language-extensions/dotnet-core-CSharp/src/managed/CSharpParamContainer.cs	Adds NUMERIC param ingestion and output replacement via SqlDecimal + struct conversion.
language-extensions/dotnet-core-CSharp/src/managed/CSharpOutputDataSet.cs	Adds result column extraction path for NUMERIC/DECIMAL into SQL_NUMERIC_STRUCT[].
language-extensions/dotnet-core-CSharp/src/managed/CSharpInputDataSet.cs	Adds input column ingestion path for NUMERIC/DECIMAL into PrimitiveDataFrameColumn<SqlDecimal>.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

InitializeColumn currently hardcodes decimalDigits=0. This new NUMERIC specialization updates columnSize to be precision, but still cannot propagate the column scale to InitColumn, so nullable NUMERIC/DECIMAL columns (especially all-NULL columns) will lose declared scale metadata. Consider overloading InitializeColumn (or calling sm_initColumnFuncPtr directly here) to pass decimalDigits extracted from the first non-NULL SQL_NUMERIC_STRUCT as well.

Suggested change

	// For NUMERIC columns, extract precision from the first non-NULL value in the column
	// columnSize for NUMERIC represents precision (1-38), not bytes
	//
	SQLULEN precision = SqlDecimalMaxPrecision; // default to SQL Server max precision
	const SQL_NUMERIC_STRUCT* columnData =
	static_cast<const SQL_NUMERIC_STRUCT*>(columnInfo->m_dataSet[columnNumber]);
	SQLINTEGER* strLenOrInd = columnInfo->m_strLen_or_Ind[columnNumber];
	// Find first non-NULL value to get precision
	//
	for (SQLULEN row = 0; row < ColumnInfo<SQL_NUMERIC_STRUCT>::sm_rowsNumber; ++row)
	{
	if (strLenOrInd[row] != SQL_NULL_DATA)
	{
	precision = columnData[row].precision;
	break;
	}
	}
	InitializeColumn(columnNumber,
	columnInfo->m_columnNames[columnNumber],
	SQL_C_NUMERIC,
	precision,
	// For NUMERIC columns, extract precision and scale from the first non-NULL value in the column
	// columnSize for NUMERIC represents precision (1-38), not bytes
	//
	SQLULEN precision = SqlDecimalMaxPrecision; // default to SQL Server max precision
	SQLSMALLINT scale = 0; // default scale if no non-NULL value is found
	const SQL_NUMERIC_STRUCT* columnData =
	static_cast<const SQL_NUMERIC_STRUCT*>(columnInfo->m_dataSet[columnNumber]);
	SQLINTEGER* strLenOrInd = columnInfo->m_strLen_or_Ind[columnNumber];
	// Find first non-NULL value to get precision and scale
	//
	for (SQLULEN row = 0; row < ColumnInfo<SQL_NUMERIC_STRUCT>::sm_rowsNumber; ++row)
	{
	if (strLenOrInd[row] != SQL_NULL_DATA)
	{
	precision = columnData[row].precision;
	scale = columnData[row].scale;
	break;
	}
	}
	// Call the underlying init-column function directly so we can pass the correct scale
	(*sm_initColumnFuncPtr)(
	columnNumber,
	columnInfo->m_columnNames[columnNumber].c_str(),
	SQL_C_NUMERIC,
	precision,
	scale,

[Copilot]

ExtractNumericColumn derives target precision as max(value.Precision) and target scale as max(value.Scale) independently. If a column contains values with different scales, increasing the chosen scale can require a larger precision for values with large integer parts (e.g., 123 + scale 2 => 123.00 needs precision 5), which will make FromSqlDecimal throw on conversion. Consider computing targetPrecision as max((value.Precision - value.Scale) + targetScale) across non-null values (where targetScale is the chosen max scale), so all values remain representable after scale normalization.

[Copilot]

using System.Linq; appears unused in this file. Removing it will avoid unnecessary dependencies/warnings and keep the helper focused.

Suggested change

	using System.Linq;
	using System.Runtime.InteropServices;
	using System.Runtime.InteropServices;

[Copilot]

PR description says SqlNumericHelper introduces additional pointer/GCHandle helpers (e.g., ToSqlNumericStructPointer / GetSqlNumericStructPointer) and centralizes pinning logic, but this implementation currently exposes only ToSqlDecimal/FromSqlDecimal/ToSqlDecimalFromPointer and the pinning is implemented in CSharpParamContainer. Either update the PR description to match the code, or add the missing helper methods and switch call sites to use them to keep the interop pattern consistent.

[Copilot]

CreateNumericStruct takes a long long mantissa and then converts it to bytes. Several new tests pass mantissas larger than LLONG_MAX (e.g., 12345678901234567890ULL), which will overflow/narrow before conversion and produce incorrect SQL_NUMERIC_STRUCT values. Consider changing mantissa to an unsigned 128-bit type (e.g., unsigned __int128) or accepting a 16-byte/4xUInt32 representation so full DECIMAL(38,*) ranges can be represented without overflow; also avoid std::abs on signed min values.

Suggested change

	inline SQL_NUMERIC_STRUCT CreateNumericStruct(
	long long mantissa,
	SQLCHAR precision,
	SQLSCHAR scale,
	bool isNegative)
	{
	// Zero-initialize all fields for safety
	SQL_NUMERIC_STRUCT result{};
	result.precision = precision;
	result.scale = scale;
	result.sign = isNegative ? 0 : 1; // 0 = negative, 1 = positive (ODBC convention)
	// Convert mantissa to little-endian byte array in val[0..15]
	// Use std::abs for long long (not plain abs which is for int)
	unsigned long long absMantissa = static_cast<unsigned long long>(std::abs(mantissa));
	// Extract bytes in little-endian order
	// Use sizeof for self-documenting code instead of magic number 16
	for (size_t i = 0; i < sizeof(result.val); i++)
	{
	result.val[i] = static_cast<SQLCHAR>(absMantissa & 0xFF);
	absMantissa >>= 8;
	}
	return result;
	}
	// Core helper that takes an unsigned 128-bit mantissa magnitude and fills SQL_NUMERIC_STRUCT.
	inline SQL_NUMERIC_STRUCT CreateNumericStruct(
	unsigned __int128 mantissa,
	SQLCHAR precision,
	SQLSCHAR scale,
	bool isNegative)
	{
	// Zero-initialize all fields for safety
	SQL_NUMERIC_STRUCT result{};
	result.precision = precision;
	result.scale = scale;
	// 0 = negative, 1 = positive (ODBC convention)
	result.sign = isNegative ? 0 : 1;
	// Convert mantissa to little-endian byte array in val[0..15]
	unsigned __int128 value = mantissa;
	// Extract bytes in little-endian order
	for (size_t i = 0; i < sizeof(result.val); i++)
	{
	result.val[i] = static_cast<SQLCHAR>(value & static_cast<unsigned __int128>(0xFF));
	value >>= 8;
	}
	return result;
	}
	// Overload for unsigned 64-bit mantissa to avoid narrowing from ULL literals.
	inline SQL_NUMERIC_STRUCT CreateNumericStruct(
	unsigned long long mantissa,
	SQLCHAR precision,
	SQLSCHAR scale,
	bool isNegative)
	{
	unsigned __int128 wideMantissa = static_cast<unsigned __int128>(mantissa);
	return CreateNumericStruct(wideMantissa, precision, scale, isNegative);
	}
	// Overload for signed 64-bit mantissa, preserving existing signature while avoiding std::abs issues.
	inline SQL_NUMERIC_STRUCT CreateNumericStruct(
	long long mantissa,
	SQLCHAR precision,
	SQLSCHAR scale,
	bool isNegative)
	{
	unsigned __int128 magnitude;
	if (mantissa < 0)
	{
	// Compute |mantissa| safely even for LLONG_MIN:
	// -(mantissa + 1) is in range, then add 1 in unsigned domain.
	long long oneLess = mantissa + 1;
	long long negOneLess = -oneLess;
	magnitude = static_cast<unsigned __int128>(static_cast<unsigned long long>(negOneLess)) + 1u;
	}
	else
	{
	magnitude = static_cast<unsigned __int128>(static_cast<unsigned long long>(mantissa));
	}
	return CreateNumericStruct(magnitude, precision, scale, isNegative);
	}