Thursday, August 20, 2020

Microsoft's Fakes and Shims

If you do unit testing you know you need to control the test environment by mocking functionality. For example, if a test requires a certain configuration, you can replace the code that normally gets the configuration from your database with code that simply returns the value the test needs.

Microsoft has mocking functionality called Fakes (which replaces the entire class) and Shims (which replaces selected methods). It is only available in the Enterprise version of Visual Studio. It's available in .Net Framework 4.0 and later and also in .Net Core with Visual Studio 2019 version 16.7 and later.

https://developercommunity.visualstudio.com/content/problem/956128/support-microsoft-fakes-on-net-core.html

Fire up your Enterprise Visual Studio 2019 and start a C#, .Net Framework, console project called StockPriceConsole.

The first thing to do is add a Class Library project called StockPrice. This is the class we will be testing. We start by adding an interface file to the StockPrice project (this is a common technique when mocking) called IStockPrice. It defines two methods and looks like this.


using System;

namespace StockPrice
{
    public interface IStockPrice
    {
        Boolean IsSymbolValid(String Symbol);
        Decimal GetStockPrice(String Symbol);
    }
}

Next we add the production implementation of IStockPrice to the StockPrice project. It implements the two methods defined in the interface. For our purposes a stock symbol must have four characters and the stock price is a random number less than 100. It looks like this


using System;

namespace StockPrice
{
    public class StockPrice : IStockPrice
    {
        public decimal GetStockPrice(string Symbol)
        {
            Random random = new Random();
            if (IsSymbolValid(Symbol))
                return (Decimal)Math.Round(random.NextDouble() * 100, 2);
            else
                throw new ArgumentException("Invalid Symbol");
        }

        public bool IsSymbolValid(string Symbol)
        {
            return (Symbol.Length == 4);
        }
    }
}

Now that we have an implementation of IStockPrice we can run it from the console. Make program.cs in StockPriceConsole look like this.


using System;

namespace StockPrice
{
    class Program
    {
        static void Main(string[] args)
        {
            StockPrice stockPrice = new StockPrice();
            String symbol = "CNET";

            Console.WriteLine(string.Format("Stock price for {0} is {1}", symbol, stockPrice.GetStockPrice(symbol)));

            Console.ReadLine();
        }
    }
}

When you run this you get output that looks something like this.


It's time to start faking it. Add a Class Library project called StockPriceTest and add a reference to StockPrice. Right-click the Reference and select Add Fakes Assembly (see below). Remember, this will only show up in the Enterprise version of Visual Studio.


You will now have a new folder called Fakes containing an XML file called StockPrice.fakes and a new reference called StockPrice.Fakes

If you open the StockPrice.fakes file you will see there's not much to it. If you have problems generating the fakes or shims you can add a Diagnostic attribute to generate useful error messages during the build.

<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/" Diagnostic="true">
  <Assembly Name="StockPrice"/>
</Fakes>

You can tell if the fakes and shims were correctly generated by double-clicking on the StockPrice.Fakes reference and then expanding it. You should see something like this. If the StockPrice.Fakes node has no children, try the Diagnostics trick explained above, rebuild, and look in the Output window.


We will start by looking at Fakes which allows you to substitute an entire class. Any methods you don't redefine are still callable but do nothing and return the default value for their return type ie zero, empty string, or null. In the example below we will redefine both methods of the IStockPrice class.


using System;
using Microsoft.QualityTools.Testing.Fakes;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using StockPrice;

namespace StockPriceTest
{
    [TestClass]
    public class GetStockPriceTest
    {
        [TestMethod]
        public void CheckIBMStockPriceWithFake()
        {
            // Uses fakes to replace the entire StockPrice class
            IStockPrice stockPrice = new StockPrice.Fakes.StubIStockPrice()
            {
                IsSymbolValidString = (symbol) => { return true; },
                GetStockPriceString = (symbol) => { return 100; }
            };

            Decimal IBMStockPrice = stockPrice.GetStockPrice("IBM");

            Assert.AreEqual(100, IBMStockPrice, "IBM stock price should be 100");
        }
    }
}

This test passes because we redefined IsSymbolValid to always be true and GetStockPrice to always return 100. Note the names of the Fakes - they have their argument type(s) appended to them so we can redefine IsSymbolValue(String) separately from IsSymbolValue(Object) etc.

As I said, when you Fake a class, any methods you don't override do nothing. Shims allow you to override some methods but leave others intact. Add another test method to StockPriceTest.cs like this. Note all Shimming must take place within a ShimsContext.

        [TestMethod]
        public void CheckIBMStockPriceWithShim()
        {
            // Uses a shim to replace the IsSymbolValid method only
            using (ShimsContext.Create()) 
            {
                StockPrice.Fakes.ShimStockPrice.AllInstances.IsSymbolValidString = (methodName, symbol) => { return true; };
                IStockPrice stockPrice = new StockPrice.StockPrice();

                Decimal IBMStockPrice = stockPrice.GetStockPrice("IBM");

                Assert.IsTrue((IBMStockPrice < 100), "Invalid stock price");
            }
        }

This test replaces IsSymbolValid so that all symbols are valid, but leaves GetStockPrice intact. This is very useful when you are testing a method that calls another method in the same class. You can Shim the second method but still test the first one.

Your test results should look like this.


Friday, August 14, 2020

Detecting if code is being run as part of a unit test

There are times when you need to know if your code is being run as part of a unit test. For example, you might want your unit test's setup to call methods that regular code is not allowed to call. That method could raise a NotImplemented exception if it is not called from a unit test. How can we do this?

In the following example I am going to setup a class, a console app that calls that class, unit tests for nUnit, xUnit, and MSTest all in the same project, and a class method that determines if it is being called from one of the unit tests.

Start a new C# console app using .Net Core in Visual Studio 2019. Call it DetectUnitTest. Add a class called TargetClass that looks like this.


namespace DetectUnitTest
{
    public static class TargetClass
    {
        public static int Add(int a, int b)
        {
            return a + b;
        }
    }
}

Now we can write Program.cs


using System;

namespace DetectUnitTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(string.Format("{0} + {1} = {2}", 2,2,TargetClass.Add(2,2)));
        }
    }
}

Run the program and see the fruits of your labor.


Let's add the unit tests. Add a new class library project called UnitTests.

To Add MSTest, use NuGet to add these three packages

  • Microsoft.Net.Test.Sdk
  • MSTest.TestAdapter
  • MSTest.TestFramework

To add nUnit, use NuGet to add these two packages

  • NUnit
  • NUnit3.TestAdapter

To Add xUnit, use NuGet to add these two packages

  • xunit
  • xunit.runner.visualstudio


Also add a reference to the DetectUnitTest project so we can test it.

Add classes to the UnitTests project called MSUnitTests, NUnitTests, and XUnitTests. We will add a simple test to each one.

MSUnitTests

using DetectUnitTest;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace UnitTests
{
    [TestClass]
    public class MSUnitTests
    {
        [TestMethod]
        public void AddTest()
        {
            Assert.AreEqual(4, TargetClass.Add(2, 2));
        }
    }
}


NUnitTests

using DetectUnitTest;
using NUnit.Framework;

namespace UnitTests
{
    class NUnitTests
    {
        [Test]
        public void AddTest()
        {
            Assert.AreEqual(4, TargetClass.Add(2, 2));
        }
    }
}


XUnitTests

using DetectUnitTest;
using Xunit;

namespace UnitTests
{
    public class XUnitTests
    {
        [Fact]
        public void AddTest()
        {
            Assert.Equal(4, TargetClass.Add(2, 2));
        }
    }
}

We can run these unit tests now. From the menu, click on Test -> Test Explorer. Then click the "Run All Tests" button (the left most green arrow). All the tests will pass.


Now we will write the method that detects if the code is being run within a unit test. Add a class to DetectUnitTest called DetectUnitTest (not very imaginative). The code caches the result which is determined by looking through the call stack to see if any of the methods are decorated with specific attributes. The idea is taken from StackOverflow, but I can't find the original post to attribute it. 

using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace DetectUnitTest
{
    public static class DetectUnitTest
    {
        public static readonly List<string> UnitTestAttributes = new List<string>
        {
            "Xunit.FactAttribute",
            "Xunit.TheoryAttribute",
            "Nunit.TestAttribute",
            "Nunit.TheoryAttribute",
            "Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute",
            "Microsoft.VisualStudio.TestTools.UnitTesting.DataTestMethodAttribute"
        };

        private static bool? _IsRunningInUnitTest = null;
        public static bool IsRunningInUnitTest
        {
            get
            {
                if (!_IsRunningInUnitTest.HasValue)
                {
                    _IsRunningInUnitTest = false;
                    foreach (var f in new StackTrace().GetFrames())
                    {
                        if (f.GetMethod().GetCustomAttributes(false).Any(x => UnitTestAttributes.Contains(x.GetType().FullName)))
                        {
                            _IsRunningInUnitTest = true;
                            break;
                        }
                    }
                }

                return _IsRunningInUnitTest.Value;
            }
        }
    }
}

Let's consume it from the console app where the value returned should be false. Modify Program.cs to add this line.

using System;

namespace DetectUnitTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(string.Format("{0} + {1} = {2}", 2,2,TargetClass.Add(2,2)));
            Console.WriteLine(string.Format("Running in unit test = {0}", DetectUnitTest.IsRunningInUnitTest));
        }
    }
}

When we run the application now, the output looks like this.


Now let's see if it can detect when it's being called from the unit test classes. Add these tests to the Unit Tests.

MSUnitTests


        [DataTestMethod]
        [DataRow(true)]
        public void IsRunningUnitTest(bool Expected)
        {
            Assert.AreEqual(Expected, DetectUnitTest.DetectUnitTest.IsRunningInUnitTest);
        }

NUnitTests


        [Datapoint] bool IsTest = true;
        [Theory]
        public void IsRunningUnitTest(bool Expected)
        {
            Assert.AreEqual(Expected, DetectUnitTest.DetectUnitTest.IsRunningInUnitTest);
        }

xUnitTests


        [Theory]
        [InlineData(true)]
        public void IsRunningUnitTest(bool Expected)
        {
            Assert.Equal(Expected, DetectUnitTest.DetectUnitTest.IsRunningInUnitTest);
        }


If you run the tests in TestExplorer now you will see all these tests are passing, so we were able to detect when we were running inside a test.


You can see the test explorer was able to homologate the results from multiple test frameworks. Take a moment to understand how impressive that is.