Coding Bolt

Vectors are a fundamental part of numerical and scientific computing. In C#, the System.Numerics namespace provides a suite of SIMD-accelerated types, including the Vector<T> class. SIMD (Single Instruction, Multiple Data) allows parallel processing of data, which can significantly enhance performance by performing the same operation on multiple data points simultaneously.

What is Vector<T>?

Vector<T> is a structure that represents a vector of a specified numeric type (T). This structure is part of the System.Numerics namespace and provides a wide range of operations that benefit from SIMD support. The count of elements in a Vector<T> instance is fixed, but this count depends on the CPU architecture of the machine running the code​.

Here’s a benchmark of using Vector<T> to add two arrays element-wise along with a for loop to achieve the same:

using System;
using System.Numerics;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

public class VectorBenchmark
{
    private double[] array1;
    private double[] array2;
    private double[] result;

    [GlobalSetup]
    public void Setup()
    {
        array1 = new double[10000];
        array2 = new double[10000];
        result = new double[10000];
        var rand = new Random();
        for (int i = 0; i < array1.Length; i++)
        {
            array1[i] = rand.NextDouble();
            array2[i] = rand.NextDouble();
        }
    }

    [Benchmark]
    public void StandardLoop()
    {
        for (int i = 0; i < array1.Length; i++)
        {
            result[i] = array1[i] + array2[i];
        }
    }

    [Benchmark]
    public void SimdVector()
    {
        int length = array1.Length;
        int vectorSize = Vector<double>.Count;
        int i;
        for (i = 0; i <= length - vectorSize; i += vectorSize)
        {
            var v1 = new Vector<double>(array1, i);
            var v2 = new Vector<double>(array2, i);
            (v1 + v2).CopyTo(result, i);
        }

        for (; i < length; i++)
        {
            result[i] = array1[i] + array2[i];
        }
    }
}

public class Program
{
    public static void Main(string[] args)
    {
        var summary = BenchmarkRunner.Run<VectorBenchmark>();
    }
}

As you can see, vectorizing the operation achieved a 4X faster method.

Vectorizable Operations in C#

In C#, the Vector<T> class in the System.Numerics namespace supports a wide array of operations that can be vectorized to take advantage of SIMD (Single Instruction, Multiple Data) capabilities. Here are the primary operations that can be vectorized:

1. Arithmetic Operations:
   • Addition: Adds two vectors element-wise.
   • Subtraction: Subtracts one vector from another element-wise.
   • Multiplication: Multiplies two vectors element-wise.
   • Division: Divides one vector by another element-wise.
2. Bitwise Operations:
   • And: Performs a bitwise AND operation on each pair of elements.
   • Or: Performs a bitwise OR operation on each pair of elements.
   • Xor: Performs a bitwise XOR operation on each pair of elements.
   • Not: Performs a bitwise NOT operation on each element.
3. Comparison Operations:
   • GreaterThan: Compares each pair of elements to determine if elements in the first vector are greater than those in the second vector.
   • GreaterThanOrEqual: Compares each pair of elements to determine if elements in the first vector are greater than or equal to those in the second vector.
   • LessThan: Compares each pair of elements to determine if elements in the first vector are less than those in the second vector.
   • LessThanOrEqual: Compares each pair of elements to determine if elements in the first vector are less than or equal to those in the second vector.
   • Equal: Compares each pair of elements to determine if they are equal.
   • NotEqual: Compares each pair of elements to determine if they are not equal.
4. Logical Operations:
   • ConditionalSelect: Selects elements from two vectors based on a condition vector.
5. Mathematical Operations:
   • Abs: Computes the absolute value of each element.
   • SquareRoot: Computes the square root of each element.
   • Ceiling: Computes the ceiling of each element.
   • Floor: Computes the floor of each element.
   • Round: Rounds each element to the nearest integer.
   • Truncate: Truncates each element to an integer.
   • Max: Computes the maximum of each pair of elements.
   • Min: Computes the minimum of each pair of elements.
6. Conversion Operations:
   • Narrowing: Narrows elements from a larger type to a smaller type.
   • Widening: Widens elements from a smaller type to a larger type.
7. Transcendental Operations (requires additional libraries or specific implementations):
   • Sin: Computes the sine of each element.
   • Cos: Computes the cosine of each element.
   • Tan: Computes the tangent of each element.
   • Exp: Computes the exponent of each element.
   • Log: Computes the natural logarithm of each element.

Here are some examples of these operations that you can use:

public class VectorExample
{
    public static void Example()
    {
        var v1 = new Vector<float>(new float[] { 1, 2, 3, 4, 5, 6, 7, 8 });
        var v2 = new Vector<float>(new float[] { 9, 10, 11, 12, 13, 14, 15, 16 });

        // Addition
        var addResult = v1 + v2;

        // Subtraction
        var subResult = v1 - v2;

        // Multiplication
        var mulResult = v1 * v2;

        // Division
        var divResult = v1 / v2;

        // Bitwise And
        var andResult = v1 & v2;

        // Bitwise Or
        var orResult = v1 | v2;

        // Greater Than
        var gtResult = Vector.GreaterThan(v1, v2);

        // Absolute value
        var absResult = Vector.Abs(v1);

        // Square Root
        var sqrtResult = Vector.SquareRoot(v1);

        Console.WriteLine("Addition Result: " + addResult);
        Console.WriteLine("Square Root Result: " + sqrtResult);
    }
}

Hope this vector cheat sheet helped.

Why Vectors Improve Performance

1. Parallel Processing: SIMD allows the execution of a single instruction on multiple data points simultaneously. This parallelism is intrinsic to the Vector<T> operations and leads to substantial performance improvements in scenarios involving large datasets.
2. Efficient Use of CPU Resources: SIMD instructions reduce the number of instructions the CPU needs to decode and execute. This not only speeds up processing but also optimizes CPU cache usage​.
3. Reduced Code Complexity: The Vector<T> class abstracts many of the complexities involved in parallel processing, providing a simple API for performing complex numerical operations​.

Practical Benefits

• Matrix Operations: Vectors and matrices in the System.Numerics namespace allow for efficient mathematical computations. For example, multiplying matrices using SIMD can significantly reduce computation time compared to traditional methods.
• Machine Learning: Vectors are crucial in machine learning algorithms for operations like dot products, vector addition, and other linear algebra computations. The performance improvements can lead to faster training and inference times​.
• Game Development: In game development, vectors are used extensively for physics calculations, animations, and rendering. The performance gains from using SIMD-accelerated vectors translate to smoother and more responsive gameplay experiences​.
• Complex Algorithms: When you need to calculate the results for a large amount of numbers, Vectors are a life saver when it comes to performance.

Conclusion

Vectors in C#, particularly the Vector<T> class from the System.Numerics namespace, have brought significant performance enhancements to the .NET ecosystem. By leveraging SIMD instructions, these vectors allow for parallel data processing, leading to faster and more efficient numerical computations. This performance boost is beneficial across various domains, including scientific computing, machine learning, and game development.

Affiliate promo

If you love learning new stuff and want to support me, consider buying a course from Dometrain using this link: Browse courses – Dometrain. Thank you!