评估Tensorflow操作在循环中非常缓慢

Question

我试图通过编码一些简单的问题来学习tensorflow：我试图找到使用直接采样蒙特卡罗方法的pi的值。

运行时间比我认为使用for loop这样做的时间要长得多。我见过其他类似的东西，我试着按照解决方案，但我认为我仍然必须做错的事情。

附在下面是我的代码：

import tensorflow as tf 
import numpy as np 
import time 

n_trials = 50000 

tf.reset_default_graph() 


x = tf.random_uniform(shape=(), name='x') 
y = tf.random_uniform(shape=(), name='y') 
r = tf.sqrt(x**2 + y**2) 

hit = tf.Variable(0, name='hit') 

# perform the monte carlo step 
is_inside = tf.cast(tf.less(r, 1), tf.int32) 
hit_op = hit.assign_add(is_inside) 

with tf.Session() as sess: 
    init_op = tf.global_variables_initializer() 
    sess.run(init_op) 

    # Make sure no new nodes are added to the graph 
    sess.graph.finalize() 

    start = time.time() 

    # Run monte carlo trials -- This is very slow 
    for _ in range(n_trials): 
     sess.run(hit_op) 

    hits = hit.eval() 
    print("Pi is {}".format(4*hits/n_trials)) 
    print("Tensorflow operation took {:.2f} s".format((time.time()-start))) 

>>> Pi is 3.15208 
>>> Tensorflow operation took 8.98 s 

相比较而言，在做numpy的一个for loop类型的解决方案是一个数量级的速度更快

start = time.time() 
hits = [ 1 if np.sqrt(np.sum(np.square(np.random.uniform(size=2)))) < 1 else 0 for _ in range(n_trials) ] 
a = 0 
for hit in hits: 
    a+=hit 
print("numpy operation took {:.2f} s".format((time.time()-start))) 
print("Pi is {}".format(4*a/n_trials)) 

>>> Pi is 3.14032 
>>> numpy operation took 0.75 s 

附在下面是整体的差异的曲线图执行不同次数的试验。

请注意：我的问题不是关于“如何执行这一任务最快的”，我承认有计算圆周率的更有效的方法。我只用它作为基准测试工具来检查tensorflow对我熟悉的东西（numpy）的性能。

Answer 1

速度缓慢已经得到了在sess.run Python和Tensorflow之间的一些通信开销，这是执行你的循环内多次做。我建议使用tf.while_loop在Tensorflow中执行计算。这将是一个更好的比较numpy。

import tensorflow as tf 
import numpy as np 
import time 

n_trials = 50000 

tf.reset_default_graph() 

hit = tf.Variable(0, name='hit') 

def body(ctr): 
    x = tf.random_uniform(shape=[2], name='x') 
    r = tf.sqrt(tf.reduce_sum(tf.square(x)) 
    is_inside = tf.cond(tf.less(r,1), lambda: tf.constant(1), lambda: tf.constant(0)) 
    hit_op = hit.assign_add(is_inside) 
    with tf.control_dependencies([hit_op]): 
     return ctr + 1 

def condition(ctr): 
    return ctr < n_trials 

with tf.Session() as sess: 
    tf.global_variables_initializer().run() 
    result = tf.while_loop(condition, body, [tf.constant(0)]) 

    start = time.time() 
    sess.run(result) 

    hits = hit.eval() 
    print("Pi is {}".format(4.*hits/n_trials)) 
    print("Tensorflow operation took {:.2f} s".format((time.time()-start))) 

Answer 2

很简单，session.run有很多开销，并且它不是以这种方式使用的。通常，例如，一个神经网络，你可以称之为一个会话。运行十几个大矩阵的乘法运算，然后这0.2毫秒所需要的根本就不重要。 至于你的情况，你可能想要类似的东西。它比我的机器上的numpy版本运行速度快5倍。

顺便说一句，你在numpy中做同样的事情。如果你使用loop来减少而不是np.sum，它会慢得多。

import tensorflow as tf 
    import numpy as np 
    import time 

    n_trials = 50000 

    tf.reset_default_graph() 

    x = tf.random_uniform(shape=(n_trials,), name='x') 
    y = tf.random_uniform(shape=(), name='y') 
    r = tf.sqrt(x**2 + y**2) 

    hit = tf.Variable(0, name='hit') 

    # perform the monte carlo step 
    is_inside = tf.cast(tf.less(r, 1), tf.int32) 
    hit2= tf.reduce_sum(is_inside) 
     #hit_op = hit.assign_add(is_inside) 

    with tf.Session() as sess: 
    # init_op = tf.global_variables_initializer() 
     sess.run(tf.initialize_all_variables()) 

     # Make sure no new nodes are added to the graph 
     sess.graph.finalize() 

     start = time.time() 

     # Run monte carlo trials -- This is very slow 
     #for _ in range(n_trials): 
     sess.run(hit2) 

     hits = hit2.eval() 
     print("Pi is {}".format(4*hits/n_trials)) 
     print("Tensorflow operation took {:.2f} s".format((time.time()-start)))