第一时间打电话与BenchmarkDotNet

我已经尝试了BenchmarkDotNet用一个简单的测试类第一时间打电话与BenchmarkDotNet

public class BenchTest 
{ 
    bool First = false; 

    [Benchmark] 
    public void FirstTimeInitEffect() 
    { 
     if (First == false) 
     { 
      Console.WriteLine("First called"); 
      First = true; 
      Thread.Sleep(1000); 
     } 
     else 
     { 
      Thread.Sleep(10); 
     } 

    } 
}

让他跟

static void Main(string[] args) 
    { 
     var cfg = ManualConfig.Create(DefaultConfig.Instance); 
     var job = new Job(); 
     job.Accuracy.RemoveOutliers = false; 
     cfg.Add(new Job[] { 
      job 
      } 
     ); 

     BenchmarkRunner.Run<BenchTest>(cfg); 
    }

这将打印一个很好的总结，但我所来自的结果无法运行第一次调用FirstTimeInitEffect方法需要1秒。我希望从基准测试工具中可以看到第一个通话效果。我已经尝试过一个自定义配置，以防止删除异常值，但这并没有把戏。我是否使用了错误的工具，或者这是否超出了工具的范围？

// * Detailed results * 
BenchTest.FirstTimeInitEffect: Job-LQPFTL(RemoveOutliers=False) 
Runtime = Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0; GC = Concurrent Workstation 
Mean = 10.8548 ms, StdErr = 0.0169 ms (0.16%); N = 15, StdDev = 0.0654 ms 
Min = 10.7158 ms, Q1 = 10.8058 ms, Median = 10.8963 ms, Q3 = 10.9011 ms, Max = 10.9029 ms 
IQR = 0.0953 ms, LowerFence = 10.6628 ms, UpperFence = 11.0440 ms 
ConfidenceInterval = [10.8217 ms; 10.8879 ms] (CI 95%) 
Skewness = -1.01049139924314, Kurtosis = 2.40561202562778 


Total time: 00:00:21 (21.92 sec) 

// * Summary * 

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.2.9200.0 
Processor=Intel(R) Core(TM) i7-4770K CPU 3.50GHz, ProcessorCount=8 
Frequency=3417979 Hz, Resolution=292.5706 ns, Timer=TSC 
    [Host]  : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 
    Job-LQPFTL : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 

RemoveOutliers=False Allocated=0 B 

       Method |  Mean | StdDev | 
-------------------- |----------- |---------- | 
FirstTimeInitEffect | 10.8548 ms | 0.0654 ms |

来源

2016-12-29 Alois Kraus

BenchmarkDotNet做了很多热身迭代;它可以在我们开始收集实际测量数据之前达到稳定状态。此外，它还会执行一些其他的东西，这些东西也会调用您的方法（jitting，pilot迭代等等）。所以，第一个电话被省略了，没有被包含在摘要中。

如果您想要根据所有测量（无需接触，试验和热身迭代）获得统计数据，则应使用RunStrategy.ColdStart（而不是RunStrategy.Throughput这是默认设置）。自BenchmarkDotNet v0.10.2以来它工作正常。

源代码：

[SimpleJob(RunStrategy.ColdStart, targetCount: 5)] 
[MinColumn, MaxColumn, MeanColumn, MedianColumn] 
public class BenchTest 
{ 
    private bool firstCall; 

    [Benchmark] 
    public void FirstTimeInitEffect() 
    { 
     if (firstCall == false) 
     { 
      firstCall = true; 
      Console.WriteLine("// First call"); 
      Thread.Sleep(1000); 
     } 
     else 
      Thread.Sleep(10); 
    } 
}

测量：

Result 1: 1 op, 1000582715.59 ns, 1.0006 s/op 
Result 2: 1 op, 10190609.23 ns, 10.1906 ms/op 
Result 3: 1 op, 10164930.24 ns, 10.1649 ms/op 
Result 4: 1 op, 10604238.53 ns, 10.6042 ms/op 
Result 5: 1 op, 10420930.04 ns, 10.4209 ms/op

详细结果：

Mean = 208.4175 ms, StdErr = 198.1331 ms (95.07%); N = 5, StdDev = 443.0390 ms 
Min = 10.1163 ms, Q1 = 10.1203 ms, Median = 10.4233 ms, Q3 = 505.7118 ms, Max = 1,000.9497 ms 
IQR = 495.5915 ms, LowerFence = -733.2670 ms, UpperFence = 1,249.0991 ms 
ConfidenceInterval = [-179.9233 ms; 596.7583 ms] (CI 95%) 
Skewness = 1.07, Kurtosis = 2.08

总结：

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.2.9200.0 
Processor=Intel(R) Core(TM) i7-6700HQ CPU 2.60GHz, ProcessorCount=8 
Frequency=2531252 Hz, Resolution=395.0614 ns, Timer=TSC 
    [Host]  : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 
    Job-IQRRGS : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 

RunStrategy=ColdStart TargetCount=5 Allocated=1.64 kB 

       Method |  Mean |  StdErr |  StdDev |  Min |   Max |  Median | 
-------------------- |------------ |------------ |------------ |----------- |-------------- |----------- | 
FirstTimeInitEffect | 208.4175 ms | 198.1331 ms | 443.0390 ms | 10.1163 ms | 1,000.9497 ms | 10.4233 ms |

来源

2017-01-22 15:01:33 AndreyAkinshin

第一时间打电话与BenchmarkDotNet

回答

相关问题