如何使用Levenstein算法优化SQL Server存储过程？

Question

此过程需要10k行左右的分钟。

如何让它更快？

CREATE TABLE #TempTable 
(
     idl int, 
     idr int, 
     prof decimal(10,2), 
     sportkind nvarchar(50), 
     eventtype nvarchar(100), 
     eventname nvarchar(100), 
     eventdate datetime, 
     iscorridore bit 
) 

INSERT INTO #TempTable 
    SELECT 
     l.Id, r.Id, 
     (100.0/(1.0/l.CoeffValue + 1.0/r.CoeffValue) - 100.0), 
     r.SportKind, r.CoeffType, r.Team1 + '-' + r.Team2, r.EventDate, 
     dbo.CheckForCorridor(l.CoeffParam, l.CoeffKind, r.CoeffParam, r.CoeffKind) 
    FROM 
     Lines l, Lines r 
    WHERE 
     --l.BookmakerName != r.BookmakerName AND 
     l.EventDate = r.EventDate 
     AND l.SportKind = r.SportKind 
     AND (l.CoeffType = r.CoeffType OR (l.CoeffType is null AND r.CoeffType is null)) 
     AND dbo.CheckForFork(l.CoeffValue, r.CoeffValue) = 1 
     AND ((dbo.Levenstein(l.Team1, r.Team1) > 0.8 
      AND dbo.Levenstein(l.Team2, r.Team2) > 0.8) 
      OR (dbo.Levenstein(l.Team1, r.Team2) < 0.3 
       OR dbo.Levenstein(l.Team2, r.Team1) < 0.3) 
     ) 
     AND dbo.CheckSimpleCoeff(l.CoeffParam, l.CoeffKind, r.CoeffParam, r.CoeffKind) = 1 

INSERT INTO Forks (Name, Profit, EventDate, [Event], SportKind, EventType, LeftLine_Id, RightLine_Id, IsCorridore) 
    --поменять eventname 
    SELECT 
     eventname, prof, eventdate, eventname, sportkind, eventtype, 
     idl, idr, iscorridore 
    FROM 
     #TempTable temp 
    WHERE 
     NOT EXISTS (SELECT * FROM Forks c 
        WHERE temp.idl = c.LeftLine_Id 
        AND temp.idr = c.RightLine_Id) 

    DROP TABLE #TempTable 
END 

Levenstein是来自Simmetrics的CLR功能。

我尝试找到一对符合上述过程规则的行，然后创建新条目并将它们写入另一个表。

Answer 1

这太长了评论。

优化Levenshtein算法很困难。一种方法是在应用算法之前对结果进行预过滤。在SQL中，可以通过创建一个每个名称和一个字母都有一行的新表，然后进行连接来完成此操作。举例来说，如果你想有共通之处，那么至少三个字母的名字：

select nl.name, nl2.name 
from nameletters nl join 
    nameletters nl2 
    on nl.char = nl2.char and nl.name <> nl2.name 
group by nl.name, nl2.name 
having count(*) >= 3; 

，或类似地，考虑Levenshtein距离只为100名拥有最字母共同点：

select * 
from (select nl.name, nl2.name, rank(count(*)) over (partition by nl.name) as rnk 
     from nameletters nl join 
      nameletters nl2 
      on nl.char = nl2.char and nl.name <> nl2.name 
     group by nl.name, nl2.name 
     having count(*) >= 3 
    ) nn 
having rnk >= 100; 

这些都是启发式方法，但它们可能有助于解决您的问题。