Premature ending of generator in list comprehensionGenerator Expressions vs. List ComprehensionHow do I check...
How do you funnel food off a cutting board?
Calculate the true diameter of stars from photographic plate
Is there a verb that means to inject with poison?
How do I prevent a homebrew Grappling Hook feature from trivializing Tomb of Annihilation?
Why did the villain in the first Men in Black movie care about Earth's Cockroaches?
After checking in online, how do I know whether I need to go show my passport at airport check-in?
False written accusations not made public - is there law to cover this?
Non-Cancer terminal illness that can affect young (age 10-13) girls?
Why does PHOTOREC keep finding files?
Has any human ever had the choice to leave Earth permanently?
Why avoid shared user accounts?
Why didn't Tom Riddle take the presence of Fawkes and the Sorting Hat as more of a threat?
Why don't key signatures indicate the tonic?
Changing the laptop's CPU. Should I reinstall Linux?
Why are the books in the Game of Thrones citadel library shelved spine inwards?
A Missing Symbol for This Logo
Does diversity provide anything that meritocracy does not?
Square Root Distance from Integers
What will happen if Parliament votes "no" on each of the Brexit-related votes to be held on the 12th, 13th and 14th of March?
Citing paid articles from illegal web sharing
Separate environment for personal and development use under macOS
Treasure Hunt Riddle
How to politely refuse in-office gym instructor for steroids and protein
What happened to my GE option?
Premature ending of generator in list comprehension
Generator Expressions vs. List ComprehensionHow do I check if a list is empty?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow to make a flat list out of list of lists?List comprehension vs mapCreate a dictionary with list comprehension in PythonHow to clone or copy a list?How do I list all files of a directory?if/else in Python's list comprehension?
I'm using generators in list comprehensions, and getting some unexpected behavior with one of the generators ending early. Why does creating the generator outside of the list comprehension cause the behavior to change?
The generator I created is as follows:
def inc_range(a,b):
for i in range(min(a,b), max(a,b) + 1):
yield i
The first way of calling is as follows:
[(i,j) for i in inc_range(1,3) for j in inc_range(4,6)]
This gives me the following result:
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The second way of calling it is as follows:
a = inc_range(1,3)
b = inc_range(4,6)
[(i,j) for i in a for j in b]
This gives me the following:
[(1, 4), (1, 5), (1, 6)]
Experimenting around, the following two examples gave me the first result:
a = range(1,4)
b = range(4,7)
[(i,j) for i in a for j in b]
a = (i for i in range(1,4))
b = (i for i in range(4,7))
a = list(a)
b = list(b)
[(i,j) for i in a for j in b]
While the following gave me the second result again.
a = (i for i in range(1,4))
b = (i for i in range(4,7))
[(i,j) for i in a for j in b]
What rule am I violating here regarding generators? Why does it make a difference when I assign the generators to variables before using them in a list comprehension, vs. using them directly?
ANSWERS
Check out the following answers which helped me understand what is occurring here:
Alex Yu
mkrieger1
python generator list-comprehension
New contributor
|
show 3 more comments
I'm using generators in list comprehensions, and getting some unexpected behavior with one of the generators ending early. Why does creating the generator outside of the list comprehension cause the behavior to change?
The generator I created is as follows:
def inc_range(a,b):
for i in range(min(a,b), max(a,b) + 1):
yield i
The first way of calling is as follows:
[(i,j) for i in inc_range(1,3) for j in inc_range(4,6)]
This gives me the following result:
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The second way of calling it is as follows:
a = inc_range(1,3)
b = inc_range(4,6)
[(i,j) for i in a for j in b]
This gives me the following:
[(1, 4), (1, 5), (1, 6)]
Experimenting around, the following two examples gave me the first result:
a = range(1,4)
b = range(4,7)
[(i,j) for i in a for j in b]
a = (i for i in range(1,4))
b = (i for i in range(4,7))
a = list(a)
b = list(b)
[(i,j) for i in a for j in b]
While the following gave me the second result again.
a = (i for i in range(1,4))
b = (i for i in range(4,7))
[(i,j) for i in a for j in b]
What rule am I violating here regarding generators? Why does it make a difference when I assign the generators to variables before using them in a list comprehension, vs. using them directly?
ANSWERS
Check out the following answers which helped me understand what is occurring here:
Alex Yu
mkrieger1
python generator list-comprehension
New contributor
unrelated - do not usefor x in ...: yield ..
- useyield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
@PatrickArtner didn't know aboutyield from
, thanks for the tip.
– bbminerva
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:j in inc_range(3,6)
vsb = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with justrange
and get results equal to "first way" or "second way". What was the point of all this? I'm confused
– Alex Yu
59 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
1
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago
|
show 3 more comments
I'm using generators in list comprehensions, and getting some unexpected behavior with one of the generators ending early. Why does creating the generator outside of the list comprehension cause the behavior to change?
The generator I created is as follows:
def inc_range(a,b):
for i in range(min(a,b), max(a,b) + 1):
yield i
The first way of calling is as follows:
[(i,j) for i in inc_range(1,3) for j in inc_range(4,6)]
This gives me the following result:
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The second way of calling it is as follows:
a = inc_range(1,3)
b = inc_range(4,6)
[(i,j) for i in a for j in b]
This gives me the following:
[(1, 4), (1, 5), (1, 6)]
Experimenting around, the following two examples gave me the first result:
a = range(1,4)
b = range(4,7)
[(i,j) for i in a for j in b]
a = (i for i in range(1,4))
b = (i for i in range(4,7))
a = list(a)
b = list(b)
[(i,j) for i in a for j in b]
While the following gave me the second result again.
a = (i for i in range(1,4))
b = (i for i in range(4,7))
[(i,j) for i in a for j in b]
What rule am I violating here regarding generators? Why does it make a difference when I assign the generators to variables before using them in a list comprehension, vs. using them directly?
ANSWERS
Check out the following answers which helped me understand what is occurring here:
Alex Yu
mkrieger1
python generator list-comprehension
New contributor
I'm using generators in list comprehensions, and getting some unexpected behavior with one of the generators ending early. Why does creating the generator outside of the list comprehension cause the behavior to change?
The generator I created is as follows:
def inc_range(a,b):
for i in range(min(a,b), max(a,b) + 1):
yield i
The first way of calling is as follows:
[(i,j) for i in inc_range(1,3) for j in inc_range(4,6)]
This gives me the following result:
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The second way of calling it is as follows:
a = inc_range(1,3)
b = inc_range(4,6)
[(i,j) for i in a for j in b]
This gives me the following:
[(1, 4), (1, 5), (1, 6)]
Experimenting around, the following two examples gave me the first result:
a = range(1,4)
b = range(4,7)
[(i,j) for i in a for j in b]
a = (i for i in range(1,4))
b = (i for i in range(4,7))
a = list(a)
b = list(b)
[(i,j) for i in a for j in b]
While the following gave me the second result again.
a = (i for i in range(1,4))
b = (i for i in range(4,7))
[(i,j) for i in a for j in b]
What rule am I violating here regarding generators? Why does it make a difference when I assign the generators to variables before using them in a list comprehension, vs. using them directly?
ANSWERS
Check out the following answers which helped me understand what is occurring here:
Alex Yu
mkrieger1
python generator list-comprehension
python generator list-comprehension
New contributor
New contributor
edited 15 mins ago
bbminerva
New contributor
asked 1 hour ago
bbminervabbminerva
434
434
New contributor
New contributor
unrelated - do not usefor x in ...: yield ..
- useyield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
@PatrickArtner didn't know aboutyield from
, thanks for the tip.
– bbminerva
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:j in inc_range(3,6)
vsb = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with justrange
and get results equal to "first way" or "second way". What was the point of all this? I'm confused
– Alex Yu
59 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
1
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago
|
show 3 more comments
unrelated - do not usefor x in ...: yield ..
- useyield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
@PatrickArtner didn't know aboutyield from
, thanks for the tip.
– bbminerva
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:j in inc_range(3,6)
vsb = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with justrange
and get results equal to "first way" or "second way". What was the point of all this? I'm confused
– Alex Yu
59 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
1
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago
unrelated - do not use
for x in ...: yield ..
- use yield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
unrelated - do not use
for x in ...: yield ..
- use yield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
@PatrickArtner didn't know about
yield from
, thanks for the tip.– bbminerva
59 mins ago
@PatrickArtner didn't know about
yield from
, thanks for the tip.– bbminerva
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:
j in inc_range(3,6)
vs b = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with just range
and get results equal to "first way" or "second way". What was the point of all this? I'm confused– Alex Yu
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:
j in inc_range(3,6)
vs b = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with just range
and get results equal to "first way" or "second way". What was the point of all this? I'm confused– Alex Yu
59 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
1
1
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago
|
show 3 more comments
3 Answers
3
active
oldest
votes
And the answer is in:
PEP 289 -- Generator Expressions. The details
Only the outermost for-expression is evaluated immediately, the other expressions are deferred until the generator is run:
g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4)
is equivalent to:
def __gen(bound_exp):
for var1 in bound_exp:
if exp2:
for var2 in exp3:
if exp4:
yield tgtexp
g = __gen(iter(exp1))
del __gen
Clarification. Why PEP-289 and not PEP-202?
Because comprehensions on lists works in another way.
This would be example for comprehensions on lists:
a = list(inc_range(1,3))
b = list(inc_range(4,6))
[(i,j) for i in a for j in b]
And there will be no "wonders": everything will be as expected.
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And usinggenerator
-functions changes way of evaluation from first to last (not outermost only)
– Alex Yu
15 mins ago
add a comment |
A generator is an iterable object hence when you call it outside of a list it returns the next item alone.
a = inc_range(1,3)
b = inc_range(4,6)
c = inc_range(7,9)
[(i,j,k) for i in a for j in b for k in c]
This will only yield the elements for k in c when run
Hence you need to iterate through all its objects as you have when defining it as an array.
[(i,j,k) for i in inc_range(1,3) for j in inc_range(3,6) for k in inc_range(7,9)]
This forces the generator to yield all values in each of the iterations.
I don't think I quite understand the example. All three are iterators, and should iterate until they raise aStopIteration
. I thought thefor i in a
would keep callinga
until aStopIteration
is thrown, much like what occurs when I turn it into a list first.
– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
add a comment |
To get the desired result, the "inner" generator would have to be run as many times as the "outer" generator yields a value.
But, after the first run, the "inner" generator is exhausted and cannot be run again.
Adding a print
illustrates this (simplifying the example):
>>> def inc(a, b):
... for i in range(a, b):
... print(i)
... yield i
...
>>> a = inc(1, 4)
>>> b = inc(4, 7)
>>> [(i, j) for i in a for j in b]
1 # <-- a begins to run
4 # <-- b begins to run
5
6 # <-- b exhausted here
2 # <-- a continued, but not resulting in list item, because lacking value from b
3
[(1, 4), (1, 5), (1, 6)]
The reason why not storing the generators in variables works as expected is because a new "inner" generator is created for each iteration of the "outer" generator. Again, illustrated by some prints:
>>> def inc(a, b):
... print('started', a, b)
... for i in range(a, b):
... yield i
...
>>> [(i, j) for i in inc(1, 4) for j in inc(4, 7)]
started 1 4
started 4 7
started 4 7
started 4 7
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The reason why using range
objects or lists works as expected is because they can be iterated over arbitrarily many times without being exhausted.
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
bbminerva is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54895114%2fpremature-ending-of-generator-in-list-comprehension%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
And the answer is in:
PEP 289 -- Generator Expressions. The details
Only the outermost for-expression is evaluated immediately, the other expressions are deferred until the generator is run:
g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4)
is equivalent to:
def __gen(bound_exp):
for var1 in bound_exp:
if exp2:
for var2 in exp3:
if exp4:
yield tgtexp
g = __gen(iter(exp1))
del __gen
Clarification. Why PEP-289 and not PEP-202?
Because comprehensions on lists works in another way.
This would be example for comprehensions on lists:
a = list(inc_range(1,3))
b = list(inc_range(4,6))
[(i,j) for i in a for j in b]
And there will be no "wonders": everything will be as expected.
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And usinggenerator
-functions changes way of evaluation from first to last (not outermost only)
– Alex Yu
15 mins ago
add a comment |
And the answer is in:
PEP 289 -- Generator Expressions. The details
Only the outermost for-expression is evaluated immediately, the other expressions are deferred until the generator is run:
g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4)
is equivalent to:
def __gen(bound_exp):
for var1 in bound_exp:
if exp2:
for var2 in exp3:
if exp4:
yield tgtexp
g = __gen(iter(exp1))
del __gen
Clarification. Why PEP-289 and not PEP-202?
Because comprehensions on lists works in another way.
This would be example for comprehensions on lists:
a = list(inc_range(1,3))
b = list(inc_range(4,6))
[(i,j) for i in a for j in b]
And there will be no "wonders": everything will be as expected.
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And usinggenerator
-functions changes way of evaluation from first to last (not outermost only)
– Alex Yu
15 mins ago
add a comment |
And the answer is in:
PEP 289 -- Generator Expressions. The details
Only the outermost for-expression is evaluated immediately, the other expressions are deferred until the generator is run:
g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4)
is equivalent to:
def __gen(bound_exp):
for var1 in bound_exp:
if exp2:
for var2 in exp3:
if exp4:
yield tgtexp
g = __gen(iter(exp1))
del __gen
Clarification. Why PEP-289 and not PEP-202?
Because comprehensions on lists works in another way.
This would be example for comprehensions on lists:
a = list(inc_range(1,3))
b = list(inc_range(4,6))
[(i,j) for i in a for j in b]
And there will be no "wonders": everything will be as expected.
And the answer is in:
PEP 289 -- Generator Expressions. The details
Only the outermost for-expression is evaluated immediately, the other expressions are deferred until the generator is run:
g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4)
is equivalent to:
def __gen(bound_exp):
for var1 in bound_exp:
if exp2:
for var2 in exp3:
if exp4:
yield tgtexp
g = __gen(iter(exp1))
del __gen
Clarification. Why PEP-289 and not PEP-202?
Because comprehensions on lists works in another way.
This would be example for comprehensions on lists:
a = list(inc_range(1,3))
b = list(inc_range(4,6))
[(i,j) for i in a for j in b]
And there will be no "wonders": everything will be as expected.
edited 30 mins ago
answered 40 mins ago
Alex YuAlex Yu
2,0011522
2,0011522
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And usinggenerator
-functions changes way of evaluation from first to last (not outermost only)
– Alex Yu
15 mins ago
add a comment |
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And usinggenerator
-functions changes way of evaluation from first to last (not outermost only)
– Alex Yu
15 mins ago
This would be the answer! Thanks!
– bbminerva
33 mins ago
This would be the answer! Thanks!
– bbminerva
33 mins ago
1
1
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
You asked excellent question. I was somewhat confused at first. What you asked - is actually question about generator-expressions not list-comprehensions
– Alex Yu
31 mins ago
2
2
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
Good findings. The important point, IMO, is that the inner expression not only is deferred, but it is evaluated again in each iteration of the outer expression.
– mkrieger1
28 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And using
generator
-functions changes way of evaluation from first to last (not outermost only)– Alex Yu
15 mins ago
@mkrieger1 As I read now Early Binding versus Late Binding there was discussion with pro- and contra- of such behaviour. And using
generator
-functions changes way of evaluation from first to last (not outermost only)– Alex Yu
15 mins ago
add a comment |
A generator is an iterable object hence when you call it outside of a list it returns the next item alone.
a = inc_range(1,3)
b = inc_range(4,6)
c = inc_range(7,9)
[(i,j,k) for i in a for j in b for k in c]
This will only yield the elements for k in c when run
Hence you need to iterate through all its objects as you have when defining it as an array.
[(i,j,k) for i in inc_range(1,3) for j in inc_range(3,6) for k in inc_range(7,9)]
This forces the generator to yield all values in each of the iterations.
I don't think I quite understand the example. All three are iterators, and should iterate until they raise aStopIteration
. I thought thefor i in a
would keep callinga
until aStopIteration
is thrown, much like what occurs when I turn it into a list first.
– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
add a comment |
A generator is an iterable object hence when you call it outside of a list it returns the next item alone.
a = inc_range(1,3)
b = inc_range(4,6)
c = inc_range(7,9)
[(i,j,k) for i in a for j in b for k in c]
This will only yield the elements for k in c when run
Hence you need to iterate through all its objects as you have when defining it as an array.
[(i,j,k) for i in inc_range(1,3) for j in inc_range(3,6) for k in inc_range(7,9)]
This forces the generator to yield all values in each of the iterations.
I don't think I quite understand the example. All three are iterators, and should iterate until they raise aStopIteration
. I thought thefor i in a
would keep callinga
until aStopIteration
is thrown, much like what occurs when I turn it into a list first.
– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
add a comment |
A generator is an iterable object hence when you call it outside of a list it returns the next item alone.
a = inc_range(1,3)
b = inc_range(4,6)
c = inc_range(7,9)
[(i,j,k) for i in a for j in b for k in c]
This will only yield the elements for k in c when run
Hence you need to iterate through all its objects as you have when defining it as an array.
[(i,j,k) for i in inc_range(1,3) for j in inc_range(3,6) for k in inc_range(7,9)]
This forces the generator to yield all values in each of the iterations.
A generator is an iterable object hence when you call it outside of a list it returns the next item alone.
a = inc_range(1,3)
b = inc_range(4,6)
c = inc_range(7,9)
[(i,j,k) for i in a for j in b for k in c]
This will only yield the elements for k in c when run
Hence you need to iterate through all its objects as you have when defining it as an array.
[(i,j,k) for i in inc_range(1,3) for j in inc_range(3,6) for k in inc_range(7,9)]
This forces the generator to yield all values in each of the iterations.
answered 1 hour ago
James CarronJames Carron
618
618
I don't think I quite understand the example. All three are iterators, and should iterate until they raise aStopIteration
. I thought thefor i in a
would keep callinga
until aStopIteration
is thrown, much like what occurs when I turn it into a list first.
– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
add a comment |
I don't think I quite understand the example. All three are iterators, and should iterate until they raise aStopIteration
. I thought thefor i in a
would keep callinga
until aStopIteration
is thrown, much like what occurs when I turn it into a list first.
– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
I don't think I quite understand the example. All three are iterators, and should iterate until they raise a
StopIteration
. I thought the for i in a
would keep calling a
until a StopIteration
is thrown, much like what occurs when I turn it into a list first.– bbminerva
53 mins ago
I don't think I quite understand the example. All three are iterators, and should iterate until they raise a
StopIteration
. I thought the for i in a
would keep calling a
until a StopIteration
is thrown, much like what occurs when I turn it into a list first.– bbminerva
53 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
@Alex Yu describes this well
– James Carron
40 mins ago
add a comment |
To get the desired result, the "inner" generator would have to be run as many times as the "outer" generator yields a value.
But, after the first run, the "inner" generator is exhausted and cannot be run again.
Adding a print
illustrates this (simplifying the example):
>>> def inc(a, b):
... for i in range(a, b):
... print(i)
... yield i
...
>>> a = inc(1, 4)
>>> b = inc(4, 7)
>>> [(i, j) for i in a for j in b]
1 # <-- a begins to run
4 # <-- b begins to run
5
6 # <-- b exhausted here
2 # <-- a continued, but not resulting in list item, because lacking value from b
3
[(1, 4), (1, 5), (1, 6)]
The reason why not storing the generators in variables works as expected is because a new "inner" generator is created for each iteration of the "outer" generator. Again, illustrated by some prints:
>>> def inc(a, b):
... print('started', a, b)
... for i in range(a, b):
... yield i
...
>>> [(i, j) for i in inc(1, 4) for j in inc(4, 7)]
started 1 4
started 4 7
started 4 7
started 4 7
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The reason why using range
objects or lists works as expected is because they can be iterated over arbitrarily many times without being exhausted.
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
add a comment |
To get the desired result, the "inner" generator would have to be run as many times as the "outer" generator yields a value.
But, after the first run, the "inner" generator is exhausted and cannot be run again.
Adding a print
illustrates this (simplifying the example):
>>> def inc(a, b):
... for i in range(a, b):
... print(i)
... yield i
...
>>> a = inc(1, 4)
>>> b = inc(4, 7)
>>> [(i, j) for i in a for j in b]
1 # <-- a begins to run
4 # <-- b begins to run
5
6 # <-- b exhausted here
2 # <-- a continued, but not resulting in list item, because lacking value from b
3
[(1, 4), (1, 5), (1, 6)]
The reason why not storing the generators in variables works as expected is because a new "inner" generator is created for each iteration of the "outer" generator. Again, illustrated by some prints:
>>> def inc(a, b):
... print('started', a, b)
... for i in range(a, b):
... yield i
...
>>> [(i, j) for i in inc(1, 4) for j in inc(4, 7)]
started 1 4
started 4 7
started 4 7
started 4 7
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The reason why using range
objects or lists works as expected is because they can be iterated over arbitrarily many times without being exhausted.
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
add a comment |
To get the desired result, the "inner" generator would have to be run as many times as the "outer" generator yields a value.
But, after the first run, the "inner" generator is exhausted and cannot be run again.
Adding a print
illustrates this (simplifying the example):
>>> def inc(a, b):
... for i in range(a, b):
... print(i)
... yield i
...
>>> a = inc(1, 4)
>>> b = inc(4, 7)
>>> [(i, j) for i in a for j in b]
1 # <-- a begins to run
4 # <-- b begins to run
5
6 # <-- b exhausted here
2 # <-- a continued, but not resulting in list item, because lacking value from b
3
[(1, 4), (1, 5), (1, 6)]
The reason why not storing the generators in variables works as expected is because a new "inner" generator is created for each iteration of the "outer" generator. Again, illustrated by some prints:
>>> def inc(a, b):
... print('started', a, b)
... for i in range(a, b):
... yield i
...
>>> [(i, j) for i in inc(1, 4) for j in inc(4, 7)]
started 1 4
started 4 7
started 4 7
started 4 7
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The reason why using range
objects or lists works as expected is because they can be iterated over arbitrarily many times without being exhausted.
To get the desired result, the "inner" generator would have to be run as many times as the "outer" generator yields a value.
But, after the first run, the "inner" generator is exhausted and cannot be run again.
Adding a print
illustrates this (simplifying the example):
>>> def inc(a, b):
... for i in range(a, b):
... print(i)
... yield i
...
>>> a = inc(1, 4)
>>> b = inc(4, 7)
>>> [(i, j) for i in a for j in b]
1 # <-- a begins to run
4 # <-- b begins to run
5
6 # <-- b exhausted here
2 # <-- a continued, but not resulting in list item, because lacking value from b
3
[(1, 4), (1, 5), (1, 6)]
The reason why not storing the generators in variables works as expected is because a new "inner" generator is created for each iteration of the "outer" generator. Again, illustrated by some prints:
>>> def inc(a, b):
... print('started', a, b)
... for i in range(a, b):
... yield i
...
>>> [(i, j) for i in inc(1, 4) for j in inc(4, 7)]
started 1 4
started 4 7
started 4 7
started 4 7
[(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]
The reason why using range
objects or lists works as expected is because they can be iterated over arbitrarily many times without being exhausted.
answered 36 mins ago
mkrieger1mkrieger1
4,62821932
4,62821932
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
add a comment |
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Thanks for the detailed explanation!
– bbminerva
22 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
Your point about "again" - is great. I remember now that I had similar confusion as OP many years ago and then forgot till today
– Alex Yu
10 mins ago
add a comment |
bbminerva is a new contributor. Be nice, and check out our Code of Conduct.
bbminerva is a new contributor. Be nice, and check out our Code of Conduct.
bbminerva is a new contributor. Be nice, and check out our Code of Conduct.
bbminerva is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54895114%2fpremature-ending-of-generator-in-list-comprehension%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
unrelated - do not use
for x in ...: yield ..
- useyield from range(min(a,b), max(a,b) + 1)
– Patrick Artner
1 hour ago
@PatrickArtner didn't know about
yield from
, thanks for the tip.– bbminerva
59 mins ago
Maybe I'm missing something. Your first way and second way are different only in one thing:
j in inc_range(3,6)
vsb = inc_range(4,6)
. Obviously you get different results. Then you make analogous operations with justrange
and get results equal to "first way" or "second way". What was the point of all this? I'm confused– Alex Yu
59 mins ago
@AlexYu that was a typo copying over, they're both inc_range(4,6)
– bbminerva
58 mins ago
1
@smac89 a typo from the question being simplified -- I'll fix it.
– bbminerva
48 mins ago