Postgresql - LEFT OUTER JOIN性能问题

这是关于postgres实际工作方式的更多理论问题。我有四张表，分别是A，B，C和D，分别为20k，870k，770k和1.5mln。 ID是主键，正在对外键进行连接（默认为索引，不允许为空）。这里是我的查询：Postgresql - LEFT OUTER JOIN性能问题

SELECT "A"."id" AS "a", 
    COUNT("B"."id") AS "b_count", 
    COUNT("C"."id") AS "c_count", 
    COUNT("D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."C_id") 
GROUP BY "A"."id"

表和查询是由django框架生成的。由于不知道的原因，它运行了很长时间，服务器崩溃了。但是，如果我将这些LEFT OUTER JOIN中的任何一个更改为INNER JOIN，则查询运行时间为4.2秒。我知道LEFT和INNER JOIN在实践中是如何工作的，但为什么在这种情况下性能差别如此之大？

附加信息：解释查询使用LEFT OUTER JOIN（EXPLAIN（分析一下，缓冲区）不起作用，因为查询没有结束）：

"GroupAggregate (cost=606144.91..628195.81 rows=20784 width=16)" 
" -> Sort (cost=606144.91..610513.52 rows=1747445 width=16)" 
"  Sort Key: A.id" 
"  -> Hash Right Join (cost=282896.74..365231.69 rows=1747445 width=16)" 
"    Hash Cond: (D.C_id = C.id)" 
"    -> Seq Scan on D (cost=0.00..32113.94 rows=1494094 width=8)" 
"    -> Hash (cost=267717.03..267717.03 rows=873257 width=12)" 
"     -> Hash Right Join (cost=208924.92..267717.03 rows=873257 width=12)" 
"       Hash Cond: (B.A_id = A.id)" 
"       -> Hash Right Join (cost=207935.28..250353.82 rows=873257 width=12)" 
"        Hash Cond: (C.B_id = B.id)" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8)" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8)" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8)" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4)" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4)"

EXPLAIN（分析一下，缓冲区）查询一个INNER JOIN代替LEFT OUTER JOIN的：

"GroupAggregate (cost=559935.67..578819.69 rows=20784 width=16) (actual time=4565.338..5090.632 rows=19567 loops=1)" 
" Buffers: shared hit=10625 read=205521, temp read=27640 written=27388" 
" -> Sort (cost=559935.67..563670.91 rows=1494094 width=16) (actual time=4565.244..4832.596 rows=1494094 loops=1)" 
"  Sort Key: A.id" 
"  Sort Method: external merge Disk: 37992kB" 
"  Buffers: shared hit=10625 read=205521, temp read=27640 written=27388" 
"  -> Hash Join (cost=278322.24..355638.06 rows=1494094 width=16) (actual time=2274.363..3341.921 rows=1494094 loops=1)" 
"    Hash Cond: (D.C_id = C.id)" 
"    Buffers: shared hit=10622 read=205521, temp read=13681 written=13429" 
"    -> Seq Scan on D (cost=0.00..32113.94 rows=1494094 width=8) (actual time=0.007..270.841 rows=1494094 loops=1)" 
"     Buffers: shared hit=3828 read=13345" 
"    -> Hash (cost=265343.13..265343.13 rows=746649 width=12) (actual time=2271.959..2271.959 rows=746649 loops=1)" 
"     Buckets: 4096 Batches: 64 Memory Usage: 512kB" 
"     Buffers: shared hit=6794 read=192176, temp read=5640 written=8351" 
"     -> Hash Join (cost=208924.92..265343.13 rows=746649 width=12) (actual time=1107.516..2138.249 rows=746649 loops=1)" 
"       Hash Cond: (B.A_id = A.id)" 
"       Buffers: shared hit=6794 read=192176, temp read=5640 written=5514" 
"       -> Hash Join (cost=207935.28..250353.82 rows=746649 width=12) (actual time=1099.799..1784.403 rows=746649 loops=1)" 
"        Hash Cond: (C.B_id = B.id)" 
"        Buffers: shared hit=6272 read=192176, temp read=5640 written=5514" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8) (actual time=0.005..203.923 rows=746649 loops=1)" 
"          Buffers: shared hit=4600 read=8973" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8) (actual time=1095.407..1095.407 rows=873453 loops=1)" 
"          Buckets: 4096 Batches: 64 Memory Usage: 547kB" 
"          Buffers: shared hit=1672 read=183203, temp written=2907" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8) (actual time=0.004..939.839 rows=873453 loops=1)" 
"           Buffers: shared hit=1672 read=183203" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4) (actual time=7.701..7.701 rows=20784 loops=1)" 
"        Buckets: 4096 Batches: 1 Memory Usage: 731kB" 
"        Buffers: shared hit=522" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4) (actual time=0.004..4.661 rows=20784 loops=1)" 
"          Buffers: shared hit=522" 
"Total runtime: 5099.046 ms"

和奖金：如果我用稍微修改查询：

SELECT "A"."id" AS "a", 
    COUNT("B"."id") AS "b_count", 
    COUNT("C"."id") AS "c_count", 
    COUNT("D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."D_id") 
WHERE "D"."id" < 1500000 -- this is always true 
GROUP BY "A"."id"

然后它也可以工作，但比使用至少一个INNER JOIN长一点点。这种情况的解释（分析，缓冲区）：

"GroupAggregate (cost=563670.91..582554.92 rows=20784 width=16) (actual time=6779.121..7286.640 rows=19567 loops=1)" 
" Buffers: shared hit=10814 read=205329, temp read=27640 written=27388" 
" -> Sort (cost=563670.91..567406.14 rows=1494094 width=16) (actual time=6777.606..7033.885 rows=1494094 loops=1)" 
"  Sort Key: A.id" 
"  Sort Method: external merge Disk: 37992kB" 
"  Buffers: shared hit=10814 read=205329, temp read=27640 written=27388" 
"  -> Hash Join (cost=278322.24..359373.29 rows=1494094 width=16) (actual time=4601.432..5674.321 rows=1494094 loops=1)" 
"    Hash Cond: (D.C_id = C.id)" 
"    Buffers: shared hit=10814 read=205329, temp read=13681 written=13429" 
"    -> Seq Scan on D (cost=0.00..35849.18 rows=1494094 width=8) (actual time=0.005..374.660 rows=1494094 loops=1)" 
"     Filter: (id < 1500000)" 
"     Buffers: shared hit=3892 read=13281" 
"    -> Hash (cost=265343.13..265343.13 rows=746649 width=12) (actual time=4600.782..4600.782 rows=746649 loops=1)" 
"     Buckets: 4096 Batches: 64 Memory Usage: 512kB" 
"     Buffers: shared hit=6922 read=192048, temp read=5640 written=8351" 
"     -> Hash Join (cost=208924.92..265343.13 rows=746649 width=12) (actual time=3363.352..4469.474 rows=746649 loops=1)" 
"       Hash Cond: (B.A_id = A.id)" 
"       Buffers: shared hit=6922 read=192048, temp read=5640 written=5514" 
"       -> Hash Join (cost=207935.28..250353.82 rows=746649 width=12) (actual time=3257.869..4066.067 rows=746649 loops=1)" 
"        Hash Cond: (C.B_id = B.id)" 
"        Buffers: shared hit=6400 read=192048, temp read=5640 written=5514" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8) (actual time=0.006..372.317 rows=746649 loops=1)" 
"          Buffers: shared hit=4664 read=8909" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8) (actual time=3257.327..3257.327 rows=873453 loops=1)" 
"          Buckets: 4096 Batches: 64 Memory Usage: 547kB" 
"          Buffers: shared hit=1736 read=183139, temp written=2907" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8) (actual time=0.004..3097.367 rows=873453 loops=1)" 
"           Buffers: shared hit=1736 read=183139" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4) (actual time=105.467..105.467 rows=20784 loops=1)" 
"        Buckets: 4096 Batches: 1 Memory Usage: 731kB" 
"        Buffers: shared hit=522" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4) (actual time=0.002..101.506 rows=20784 loops=1)" 
"          Buffers: shared hit=522" 
"Total runtime: 7294.388 ms"

为什么原始查询中断postgres，而其他两个工作像一个魅力？

编辑：即使没有count（）子句，所有查询的行为都是相同的。问题仅在于联接

来源

2016-06-07 Damian Malarczyk

http://wiki.postgresql.org/wiki/SlowQueryQuestions –

谢谢，我增加了更多的信息，以我的问题 –

因为LEFT JOIN可以将结果集乘以很多，具体取决于表内容。

首先，你的查询似乎是错误的，你也算重复，所以你应该使用COUNT(DISTINCT columnName)

SELECT "A"."id" AS "a", 
    COUNT(distinct "B"."id") AS "b_count", 
    COUNT(distinct "C"."id") AS "c_count", 
    COUNT(distinct "D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."C_id") 
GROUP BY "A"."id

其次，考虑增加适当的索引。

A - (id) 
B - (id,A_id) 
C - (id,B_id) 
D - (id,C_id)

来源

2016-06-07 10:14:41 sagi

DISTINCT因为这些ID是主键在这里不需要。索引已经存在，因为JOIN是在外键（默认为索引）上制作的 – PawelRoman

如果'ID'是主键，例如'c'表有2个记录具有相同的'b_id'，那么这并不重要。而表'd'有两个记录具有相同的'c_id'，'C.id'的计数将是4而不是2. @PawelRoman – sagi

你是对的，对不起，我的坏。尽管如此，它并没有改变我们的观点。 COUNTs根本就不是问题。如果我们删除COUNT，则会出现同样的问题：LEFT JOINS需要很长时间才能完成，而用INNER JOIN替换一个LEFT JOIN可以将查询执行时间缩短为5秒。为什么？连接正在进行外键（空值不允许，使用默认索引） – PawelRoman

Postgresql - LEFT OUTER JOIN性能问题

回答

相关问题