Postgresql：创建表时，UNION ALL比UNION慢？

我有一个UNION查询。在EXPLAIN输出中，我注意到UNION总是在追加后进行排序。所以我把它改为UNION ALL。Postgresql：创建表时，UNION ALL比UNION慢？

据说查询应该更快？但实际执行表明，使用UNION ALL查询总是在几秒后返回。（也许我的数据集不够大，差别看起来很小，但我仍然没有意义。）我错过了什么？

QUERY

WITH wcal AS (...), wlog AS (...) 
SELECT * INTO temp.utlog FROM (
    SELECT * FROM wcal 
    UNION SELECT * FROM wlog 
) as t2;

EXPLAIN UNION

QUERY PLAN 
Unique (cost=691313.04..737080.66 rows=3051175 width=80) 
    CTE wcal 
    CTE wlog 
    -> Sort (cost=691313.04..698940.98 rows=3051175 width=80) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid 
     -> Append (cost=0.00..91535.25 rows=3051175 width=80) 
       -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) 
       -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) 
        -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64)

EXPLAIN UNION ALL

QUERY PLAN 
Append (cost=0.00..91535.25 rows=3051175 width=80) 
    CTE wcal ... 
    CTE wlog ... 
    -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) 
     -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64)

实际结果;

与UNION

Table temp.utlog dropped 

Execution time: 0.02s 
Statement 1 of 5 finished 

0 rows affected 
WITH executed successfully 

Execution time: 6.05s 
Statement 2 of 5 finished

与UNION ALL

Table temp.utlog1 dropped 

Execution time: 0.02s 
Statement 1 of 5 finished 

0 rows affected 
WITH executed successfully 

Execution time: 6.81s 
Statement 2 of 5 finished

在新表

结果计数是一样的...

SELECT count(*) FROM temp.utlog1 
UNION ALL 
SELECT count(*) FROM temp.utlog 

count 
364414 
364414

EXPLAIN ANALYZEUNION ALL

QUERY PLAN 
Append (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.670..3382.220 rows=364414 loops=1) 
    CTE wcal ... 
    CTE wlog ... 
    -> CTE Scan on wcal (cost=0.00..720.00 rows=36000 width=60) (actual time=0.668..140.251 rows=33624 loops=1) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..10578.36 rows=352612 width=64) (actual time=2635.901..3179.006 rows=330790 loops=1) 
     -> CTE Scan on wlog (cost=0.00..7052.24 rows=352612 width=64) (actual time=2635.897..3043.421 rows=330790 loops=1) 
Planning time: 1.759 ms 
Execution time: 6570.793 ms 
... 
Planning time: 1.713 ms 
Execution time: 5883.275 ms 
... 
Planning time: 1.351 ms 
Execution time: 7092.713 ms 
... 
Planning time: 1.318 ms 
Execution time: 5803.567 ms

EXPLAIN ANALYZEUNION

QUERY PLAN 
Unique (cost=65006.04..70835.22 rows=388612 width=80) (actual time=4259.243..4534.226 rows=364414 loops=1) 
    CTE wcal ... 
    CTE wlog ... 
    -> Sort (cost=65006.04..65977.57 rows=388612 width=80) (actual time=4259.239..4370.691 rows=364414 loops=1) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid 
     Sort Method: external sort Disk: 11832kB 
     -> Append (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.677..3486.529 rows=364414 loops=1) 
       -> CTE Scan on wcal (cost=0.00..720.00 rows=36000 width=60) (actual time=0.676..120.320 rows=33624 loops=1) 
       -> Subquery Scan on "*SELECT* 2" (cost=0.00..10578.36 rows=352612 width=64) (actual time=2760.279..3309.084 rows=330790 loops=1) 
        -> CTE Scan on wlog (cost=0.00..7052.24 rows=352612 width=64) (actual time=2760.275..3189.444 rows=330790 loops=1) 
Planning time: 1.500 ms 
Execution time: 7577.380 ms 
... 
Planning time: 1.375 ms 
Execution time: 6777.699 ms 
... 
Planning time: 1.375 ms 
Execution time: 6777.699 ms 
... 
Planning time: 1.340 ms 
Execution time: 6964.954 ms

UNION（0._s）和UNION ALL（4。_s），而不SELECT INTO

WITH executed successfully 

Execution time: 0.41s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.28s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.33s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.25s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.28s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.67s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.17s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.17s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.33s 
Statement 1 of 5 finished

UNION而不CTE

QUERY PLAN 
Unique (cost=699529.52..752925.08 rows=3051175 width=65) (actual time=3041.962..3488.246 rows=364414 loops=1) 
    -> Sort (cost=699529.52..707157.46 rows=3051175 width=65) (actual time=3041.958..3296.505 rows=364414 loops=1) 
     Sort Key: s1.s1, ((date_trunc('day'::text, s1.s1))::date), (NULL::character varying(3)), s2.wline, (NULL::boolean), (NULL::integer) 
     Sort Method: external merge Disk: 12544kB 
     -> Append (cost=0.00..120611.73 rows=3051175 width=65) (actual time=0.729..2218.973 rows=364414 loops=1) 
       -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=36) (actual time=0.727..94.359 rows=33624 loops=1) 
       ... 
       -> WindowAgg (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.737..2049.979 rows=330790 loops=1) 
       ... 

Planning time: 1.318 ms 
Execution time: 6438.852 ms

UNION与CTE

QUERY PLAN 
Unique (cost=670453.04..723848.60 rows=3051175 width=65) (actual time=3726.279..3973.870 rows=364414 loops=1) 
    CTE wcal 
    -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=36) (actual time=1.788..82.337 rows=33624 loops=1) 
    ... 
    CTE wlog 
    -> Sort (cost=92006.40..93059.34 rows=421175 width=24) (actual time=2542.472..2743.255 rows=330790 loops=1) 
      Sort Key: tbl_line_evts.evt_time 
      Sort Method: external merge Disk: 11712kB 
      -> WindowAgg (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.932..1982.497 rows=330790 loops=1) 
      ... 
    -> Sort (cost=670453.04..678080.98 rows=3051175 width=65) (actual time=3726.275..3823.236 rows=364414 loops=1) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wdirty, wcal.wid 
     Sort Method: external sort Disk: 12584kB 
     -> Append (cost=0.00..91535.25 rows=3051175 width=65) (actual time=1.800..3082.954 rows=364414 loops=1) 
       -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=57) (actual time=1.798..125.699 rows=33624 loops=1) 
       -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=49) (actual time=2542.476..2915.524 rows=330790 loops=1) 
Planning time: 2.381 ms 
Execution time: 5950.324 ms

UNION ALL没有CTE

QUERY PLAN 
Sort (cost=721442.46..729070.40 rows=3051175 width=80) (actual time=3416.176..3627.253 rows=364414 loops=1) 
    Sort Key: s1.s1 
    Sort Method: external merge Disk: 11816kB 
    -> Append (cost=0.00..121664.67 rows=3051175 width=80) (actual time=0.675..2726.980 rows=364414 loops=1) 
     -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.674..118.694 rows=33624 loops=1) 
     ... 
     -> Subquery Scan on "*SELECT* 2" (cost=198.08..49286.79 rows=421175 width=51) (actual time=1.223..2517.292 rows=330790 loops=1) 
       -> WindowAgg (cost=198.08..45075.04 rows=421175 width=51) (actual time=1.218..2334.702 rows=330790 loops=1) 
       ... 
Planning time: 1.360 ms 
Execution time: 5795.449 ms

UNION ALL与CTE

QUERY PLAN 
Append (cost=0.00..91535.25 rows=3051175 width=80) (actual time=0.718..3595.377 rows=364414 loops=1) 
    CTE wcal 
    -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.709..86.700 rows=33624 loops=1) 
    ... 
    CTE wlog 
    -> Sort (cost=98820.34..99873.28 rows=421175 width=51) (actual time=2818.220..3049.432 rows=330790 loops=1) 
      Sort Key: tbl_line_evts.evt_time 
      Sort Method: external merge Disk: 10952kB 
      -> WindowAgg (cost=198.08..45075.04 rows=421175 width=51) (actual time=0.884..2224.477 rows=330790 loops=1) 
      ... 
    -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) (actual time=0.716..129.933 rows=33624 loops=1) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) (actual time=2818.229..3400.468 rows=330790 loops=1) 
     -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64) (actual time=2818.224..3250.670 rows=330790 loops=1) 
Planning time: 2.415 ms 
Execution time: 6223.041 ms

来源

2017-04-09 Ben

'UNION'是否减小返回集的大小？即，是否有重复？此外，该排序可能会影响INSERT期间的索引构建。 – jcaron

否定的，他们产生了相同数量的记录。你可以详细说明索引吗？ – Ben

将数据插入索引时，如果按索引顺序插入，而不是由于I/O（尤其是在旋转光盘（而不是SSD）上）的不同顺序或随机顺序，则通常会更快。在没有插入的查询时间中，Ines是UNION，哪些是UNION ALL？ – jcaron

你的假设调和docs

UNION ......从它的结果消除了重复的行，以同样的方式作为 DISTINCT，除非使用UNION ALL 。

（......）我的

如果您选中这两个查询的费用，你会发现UNION ALL价格要便宜得多。 91K vs 737K。现在来解释一下实际结果的差异，我假设这是io缓存和写入操作的结果，尝试测量时间而不创建表格，但只是运行10解释分析与UNION，然后与UNION ALL 10次解释。应该不那么令人惊讶。

来源

2017-04-09 08:31:12

我更新了结果。你是对的，没有创建表，差别很大，用ANALYZE，结果不稳定。但没有ANALYZE，有线结果仍然是可重复的。那么我应该学什么？创建io重型桌子时，我应该更喜欢'UNION'到'UNION ALL'吗？ – Ben

'explain analyze'实际上执行查询，而只是'explain'估计它。当你想要一个没有“DISTINCT”的快速联合时，你应该使用'union all';如果你想要慢得多但只有不同的行，则应该使用'union'... –

谢谢。我想我明白了。在我的情况下，如果在缓存中存在任何低效率，'union all'返回比'union'快14倍的结果，1s的差异并不是那么糟糕。 “分析”引入了更多的延迟/中断，可能会减轻强度，所以多次查看结果比单纯查询更好，但它不稳定。 – Ben

Postgresql：创建表时，UNION ALL比UNION慢？

回答

相关问题