0
我正在为节点js使用mongodb mongoose库。CPU使用mogodb命中几乎100％
我有一个包含大约20个图表的页面。所有图表都与mongo db中的相同集合有关。
我必须为相同集合上的图表调用差不多15-16个服务。
当我这样做时，cpu与MongoDB数据库服务器差不多100％。
任何人都可以建议如何优化这个。
以下是mongodb查询的日志。
2017-09-01T23:57:25.164+0000 I COMMAND [conn4] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } },
{ $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } },
{ $unwind: “$userDetail” }, { $project: { mac: “$_id.mac”, groupField: “$userDetail.city” } },
{ $group: { _id: “$groupField”, count: { $sum: 1 } } },
{ $sort: { count: -1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10810 nreturned:0 reslen:18599
locks:{ Global: { acquireCount: { r: 26198 } }, Database: { acquireCount: { r: 13099 } },
Collection: { acquireCount: { r: 13098 } } } protocol:op_query 7755ms
2017-09-01T23:57:25.264+0000 I COMMAND [conn3] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } },
{ $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } },
{ $unwind: “$userDetail” }, { $project: { mac: “$_id.mac”, groupField: “$userDetail.country” } },
{ $group: { _id: “$groupField”, count: { $sum: 1 } } }, { $sort: { count: -1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10831 nreturned:0 reslen:82
locks:{ Global: { acquireCount: { r: 26240 } }, Database: { acquireCount: { r: 13120 } },
Collection: { acquireCount: { r: 13119 } } } protocol:op_query 7776ms
2017-09-01T23:57:25.486+0000 I COMMAND [conn5] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } },
{ $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } },
{ $unwind: “$userDetail” }, { $project: { mac: “$_id.mac”, groupField: “$userDetail.state” } },
{ $group: { _id: “$groupField”, count: { $sum: 1 } } }, { $sort: { count: -1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10806 nreturned:0 reslen:2053
locks:{ Global: { acquireCount: { r: 26190 } }, Database: { acquireCount: { r: 13095 } },
Collection: { acquireCount: { r: 13094 } } } protocol:op_query 7336ms
2017-09-01T23:57:25.564+0000 I COMMAND [conn6] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } },
{ $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } },
{ $unwind: “$userDetail” }, { $project: { mac: “$_id.mac”, groupField: “$userDetail.state” } },
{ $group: { _id: “$groupField”, count: { $sum: 1 } } }, { $sort: { count: -1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10811 nreturned:0 reslen:2053
locks:{ Global: { acquireCount: { r: 26200 } }, Database: { acquireCount: { r: 13100 } },
Collection: { acquireCount: { r: 13099 } } } protocol:op_query 8259ms
2017-09-01T23:57:26.018+0000 I COMMAND [conn2] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } }, { $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } },
{ $unwind: “$userDetail” }, { $project: { mac: “$_id.mac”, groupField: “$userDetail.city” } },
{ $group: { _id: “$groupField”, count: { $sum: 1 } } }, { $sort: { count: -1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10804 nreturned:0 reslen:18599
locks:{ Global: { acquireCount: { r: 26186 } }, Database: { acquireCount: { r: 13093 } },
Collection: { acquireCount: { r: 13092 } } } protocol:op_query 7978ms
2017-09-01T23:57:29.030+0000 I COMMAND [conn5] command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1503896340000) } } },
{ $group: { _id: “$mac”, minTime: { $min: “$t” }, maxTime: { $max: “$t” } } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 numYields:10751 nreturned:0 reslen:26423
locks:{ Global: { acquireCount: { r: 21538 } }, Database: { acquireCount: { r: 10769 } },
Collection: { acquireCount: { r: 10768 } } } protocol:op_query 3537ms
2017-09-01T23:57:29.812+0000 I COMMAND [conn3] command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503982800000), $lt: new Date(1504069140000) } } },
{ $group: { _id: “$mac”, minTime: { $min: “$t” }, maxTime: { $max: “$t” } } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 numYields:10755 nreturned:0 reslen:25216
locks:{ Global: { acquireCount: { r: 21540 } }, Database: { acquireCount: { r: 10770 } },
Collection: { acquireCount: { r: 10769 } } } protocol:op_query 4540ms
2017-09-01T23:57:30.641+0000 I COMMAND [conn6] command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503896400000), $lt: new Date(1503982740000) } } },
{ $group: { _id: “$mac”, minTime: { $min: “$t” }, maxTime: { $max: “$t” } } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 numYields:10762 nreturned:0 reslen:25145
locks:{ Global: { acquireCount: { r: 21558 } }, Database: { acquireCount: { r: 10779 } },
Collection: { acquireCount: { r: 10778 } } } protocol:op_query 5077ms
2017-09-01T23:57:31.664+0000 I COMMAND [conn2] command HyperlocalPortalDB.users command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1504069140000) } } },
{ $group: { _id: { mac: “$mac” } } }, { $lookup: { from: “users”, localField: “_id.mac”, foreignField: “mac”, as: “userDetail” } },
{ $match: { userDetail.activations.activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’) } }, { $unwind: “$userDetail” },
{ $project: { mac: “$_id.mac”, groupField: “$userDetail.country” } }, { $group: { _id: “$groupField”, count: { $sum: 1 } } },
{ $sort: { count: -1 } } ] } planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10774 nreturned:0 reslen:82
locks:{ Global: { acquireCount: { r: 26126 } }, Database: { acquireCount: { r: 13063 } },
Collection: { acquireCount: { r: 13062 } } } protocol:op_query 5638ms
2017-09-01T23:57:33.839+0000 I COMMAND [conn5] warning: log line attempted (89kB) over max size (10kB), printing beginning and end ... command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”,
pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503810000000), $lt: new Date(1503896340000) }, mac: { $in: [ “41:9e:87:01:00:00", “41:9E:87:01:00:00”, “ab:cd:ed”, “41:9e:87:01:09:64”, “41:9E:87:01:09:64" ] } } },
{ $group: { _id: { $dateToString: { format: “%Y-%m-%d %H”, date: { $subtract: [ “$t”, 14400000 ] } } },
macList: { $addToSet: “$mac” } } }, { $sort: { _id: 1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10756 nreturned:0 reslen:80098
locks:{ Global: { acquireCount: { r: 21548 } }, Database: { acquireCount: { r: 10774 } },
Collection: { acquireCount: { r: 10773 } } } protocol:op_query 4433ms
2017-09-01T23:57:34.270+0000 I COMMAND [conn6] warning: log line attempted (89kB) over max size (10kB), printing beginning and end ... command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”, pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503982800000), $lt: new Date(1504069140000) }, mac: { $in: [ “41:9e:87:01:00:00”, “41:9E:87:01:00:00", “ab:cd:ed”, “AB:CD:ED”, “41:9e:87:01:09:64", “41:9E:87:01:09:64” ] } } },
{ $group: { _id: { $dateToString: { format: “%Y-%m-%d %H”, date: { $subtract: [ “$t”, 14400000 ] } } },
macList: { $addToSet: “$mac” } } }, { $sort: { _id: 1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10749 nreturned:0 reslen:57332
locks:{ Global: { acquireCount: { r: 21528 } }, Database: { acquireCount: { r: 10764 } },
Collection: { acquireCount: { r: 10763 } } } protocol:op_query 3619ms
2017-09-01T23:57:34.376+0000 I COMMAND [conn4] warning: log line attempted (89kB) over max size (10kB), printing beginning and end ... command HyperlocalPortalDB.coordinates command: aggregate { aggregate: “coordinates”, pipeline: [ { $match: { activationId: ObjectId(‘59a6e4dad95d240cf4ba37de’), t: { $gte: new Date(1503896400000), $lt: new Date(1503982740000) }, mac: { $in: [ “41:9e:87:01:00:00", “41:9E:87:01:00:00”, “ab:cd:ed”, “AB:CD:ED”, “41:9e:87:01:09:64”, “41:9E:87:01:09:64" ] } } },
{ $group: { _id: { $dateToString: { format: “%Y-%m-%d %H”, date: { $subtract: [ “$t”, 14400000 ] } } }, macList: { $addToSet: “$mac” } } },
{ $sort: { _id: 1 } } ] }
planSummary: COLLSCAN keysExamined:0 docsExamined:1373500 hasSortStage:1 numYields:10758 nreturned:0 reslen:76210
locks:{ Global: { acquireCount: { r: 21550 } }, Database: { acquireCount: { r: 10775 } },
Collection: { acquireCount: { r: 10774 } } } protocol:op_query 4973ms
你有'activationId'字段的索引吗？ – felix
由于列出的问题太多，我削减了很长的评论。如何显示您拥有的数据以及您需要的结果。同时解释如何首先获取数据。无论如何，您可能应该预先汇总您提交大量请求的任何数据。 –
@felix'慢速查询日志条目中的'COLLSCAN'说NO。索引使用只是日志管道中显示的众多问题之一。 –