خرید بک لینک

Vote count: 0

I have the following list comprehension in my Python 2.7 code which retus the line number (index) and the line from a long list of lines:

results = [[lines.index(line), line] for line in lines
            if search_item in line.lower()]

This is lightning quick if the number of results is low:

The search item is: [ 1330 ]
Before string pre-processing, the time is: 0.0000
The number of lines is: 1,028,952
After string pre-processing, the time is: 0.2500
The number of results is: 249

"String pre-processing" is what I am calling the results = operation above.

Here is the same operation but with "1330" as the search item instead of " 1330 ". This one yields 6,049 matches instead of 249:

The search item is: [1330]
Before string pre-processing, the time is: 0.0000
The number of lines is: 1,028,952
After string pre-processing, the time is: 10.3180
The number of results is: 6,049

As you can see, 10 sec vs. 1/4 sec... Furthermore, " 1330 " and "1330" searches run in 2.4 and 3.2 sec respectively using a for loop:

for lineNum, line in enumerate(lines):
    if search_item in line.lower():
        retu lineNum, line

So, list comprehension gives a 10x improvement in performance in the case of 249 results but is 3+x slower for 6,049 results...

Obviously, the issue is not in the if/in portion of list comprehension (both searches scan through all 1M+ lines and either accept or reject each one) but in constructing a results list which is "long'ish" in the second case. In other words, the bottleneck appears to be in the

results = [lines.index(line), line]

portion of the comprehension.

I guess I am very surprised that list comprehension becomes so slow for large'ish result sets (and 6K is really not that large). What am I missing? Is there a different method I should be using that would consistently outperform the for loop?

Thank you!

asked 1 min ago

برچسب: نویسنده: استخدام کار تاريخ: دوشنبه 3 خرداد 1395 ساعت: 0:16

صفحه بندی