Mathematica equivalent of NumPy broadcast addition for large arrays [duplicate]
Clash Royale CLAN TAG#URR8PPP
up vote
10
down vote
favorite
This question already has an answer here:
How to implement the general array broadcasting method from NumPy?
5 answers
What is the Mathematica equivalent of the following Python code with the vectors' broadcast addition?
import numpy as np
a = np.random.rand(5000, 1, 5);
b = np.random.rand(1, 500, 5);
result = a + b #shape: (5000, 500, 5)
list-manipulation array python
marked as duplicate by jkuczm, gwr, C. E., xzczd, Henrik Schumacher Aug 28 at 10:56
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
add a comment |Â
up vote
10
down vote
favorite
This question already has an answer here:
How to implement the general array broadcasting method from NumPy?
5 answers
What is the Mathematica equivalent of the following Python code with the vectors' broadcast addition?
import numpy as np
a = np.random.rand(5000, 1, 5);
b = np.random.rand(1, 500, 5);
result = a + b #shape: (5000, 500, 5)
list-manipulation array python
marked as duplicate by jkuczm, gwr, C. E., xzczd, Henrik Schumacher Aug 28 at 10:56
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
2
Maybe one can play around withOuter
.
â Henrik Schumacher
Aug 27 at 12:37
1
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast.broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.
â jkuczm
Aug 28 at 10:08
add a comment |Â
up vote
10
down vote
favorite
up vote
10
down vote
favorite
This question already has an answer here:
How to implement the general array broadcasting method from NumPy?
5 answers
What is the Mathematica equivalent of the following Python code with the vectors' broadcast addition?
import numpy as np
a = np.random.rand(5000, 1, 5);
b = np.random.rand(1, 500, 5);
result = a + b #shape: (5000, 500, 5)
list-manipulation array python
This question already has an answer here:
How to implement the general array broadcasting method from NumPy?
5 answers
What is the Mathematica equivalent of the following Python code with the vectors' broadcast addition?
import numpy as np
a = np.random.rand(5000, 1, 5);
b = np.random.rand(1, 500, 5);
result = a + b #shape: (5000, 500, 5)
This question already has an answer here:
How to implement the general array broadcasting method from NumPy?
5 answers
list-manipulation array python
edited Aug 28 at 10:17
gwr
6,80122356
6,80122356
asked Aug 27 at 12:06
Maria Sargsyan
1085
1085
marked as duplicate by jkuczm, gwr, C. E., xzczd, Henrik Schumacher Aug 28 at 10:56
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
marked as duplicate by jkuczm, gwr, C. E., xzczd, Henrik Schumacher Aug 28 at 10:56
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
2
Maybe one can play around withOuter
.
â Henrik Schumacher
Aug 27 at 12:37
1
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast.broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.
â jkuczm
Aug 28 at 10:08
add a comment |Â
1
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
2
Maybe one can play around withOuter
.
â Henrik Schumacher
Aug 27 at 12:37
1
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast.broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.
â jkuczm
Aug 28 at 10:08
1
1
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
2
2
Maybe one can play around with
Outer
.â Henrik Schumacher
Aug 27 at 12:37
Maybe one can play around with
Outer
.â Henrik Schumacher
Aug 27 at 12:37
1
1
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast. broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.â jkuczm
Aug 28 at 10:08
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast. broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.â jkuczm
Aug 28 at 10:08
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
8
down vote
For your specific case (dimension 1 only in the first two slots), this might work:
a = RandomReal[0, 1, 5000, 1, 5];
b = RandomReal[0, 1, 1, 500, 5];
c1 = Flatten[
Outer[Plus, a, b, 2],
1, 3, 2, 4
]// RepeatedTiming // First
0.255
It is a bit more tedious to use Compile
but also a bit faster:
Creating the CompiledFunction
s:
cf = Compile[a, _Real, 2,
Table[Flatten[a, 1], 500],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
cg = Compile[b, _Real, 3,
Table[Flatten[b, 1], 5000],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
Running the actual code:
c2 = Plus[cf[a], cg[b]]; // RepeatedTiming // First
Max[Abs[c1 - c2]]
0.19
0.
Final remarks
The general case may be treated by a suitable combination of ArrayReshape
, MapThread
, Outer
, and Flatten
. Or, maybe even better, by ad-hoc compliled, Listable
CompiledFunction
s such as cf
and cg
instead of MapThread
. Anyways, one would probably need a thourough case analysis for that.
add a comment |Â
up vote
3
down vote
You can use:
a = RandomReal[0,1, 3,5]
b = RandomReal[0,1, 8,5]
c = Table[a[[i]] + b[[j]], i, Length[a], j, Length[b]]
Dimension[c]
will be 3,8,5
, similar to result.shape
in Python of (3,8,5)
5
In this simplified case, alsoOuter[Plus, a, b, 1]
works (and should be much faster).
â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know aboutOuter
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
8
down vote
For your specific case (dimension 1 only in the first two slots), this might work:
a = RandomReal[0, 1, 5000, 1, 5];
b = RandomReal[0, 1, 1, 500, 5];
c1 = Flatten[
Outer[Plus, a, b, 2],
1, 3, 2, 4
]// RepeatedTiming // First
0.255
It is a bit more tedious to use Compile
but also a bit faster:
Creating the CompiledFunction
s:
cf = Compile[a, _Real, 2,
Table[Flatten[a, 1], 500],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
cg = Compile[b, _Real, 3,
Table[Flatten[b, 1], 5000],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
Running the actual code:
c2 = Plus[cf[a], cg[b]]; // RepeatedTiming // First
Max[Abs[c1 - c2]]
0.19
0.
Final remarks
The general case may be treated by a suitable combination of ArrayReshape
, MapThread
, Outer
, and Flatten
. Or, maybe even better, by ad-hoc compliled, Listable
CompiledFunction
s such as cf
and cg
instead of MapThread
. Anyways, one would probably need a thourough case analysis for that.
add a comment |Â
up vote
8
down vote
For your specific case (dimension 1 only in the first two slots), this might work:
a = RandomReal[0, 1, 5000, 1, 5];
b = RandomReal[0, 1, 1, 500, 5];
c1 = Flatten[
Outer[Plus, a, b, 2],
1, 3, 2, 4
]// RepeatedTiming // First
0.255
It is a bit more tedious to use Compile
but also a bit faster:
Creating the CompiledFunction
s:
cf = Compile[a, _Real, 2,
Table[Flatten[a, 1], 500],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
cg = Compile[b, _Real, 3,
Table[Flatten[b, 1], 5000],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
Running the actual code:
c2 = Plus[cf[a], cg[b]]; // RepeatedTiming // First
Max[Abs[c1 - c2]]
0.19
0.
Final remarks
The general case may be treated by a suitable combination of ArrayReshape
, MapThread
, Outer
, and Flatten
. Or, maybe even better, by ad-hoc compliled, Listable
CompiledFunction
s such as cf
and cg
instead of MapThread
. Anyways, one would probably need a thourough case analysis for that.
add a comment |Â
up vote
8
down vote
up vote
8
down vote
For your specific case (dimension 1 only in the first two slots), this might work:
a = RandomReal[0, 1, 5000, 1, 5];
b = RandomReal[0, 1, 1, 500, 5];
c1 = Flatten[
Outer[Plus, a, b, 2],
1, 3, 2, 4
]// RepeatedTiming // First
0.255
It is a bit more tedious to use Compile
but also a bit faster:
Creating the CompiledFunction
s:
cf = Compile[a, _Real, 2,
Table[Flatten[a, 1], 500],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
cg = Compile[b, _Real, 3,
Table[Flatten[b, 1], 5000],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
Running the actual code:
c2 = Plus[cf[a], cg[b]]; // RepeatedTiming // First
Max[Abs[c1 - c2]]
0.19
0.
Final remarks
The general case may be treated by a suitable combination of ArrayReshape
, MapThread
, Outer
, and Flatten
. Or, maybe even better, by ad-hoc compliled, Listable
CompiledFunction
s such as cf
and cg
instead of MapThread
. Anyways, one would probably need a thourough case analysis for that.
For your specific case (dimension 1 only in the first two slots), this might work:
a = RandomReal[0, 1, 5000, 1, 5];
b = RandomReal[0, 1, 1, 500, 5];
c1 = Flatten[
Outer[Plus, a, b, 2],
1, 3, 2, 4
]// RepeatedTiming // First
0.255
It is a bit more tedious to use Compile
but also a bit faster:
Creating the CompiledFunction
s:
cf = Compile[a, _Real, 2,
Table[Flatten[a, 1], 500],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
cg = Compile[b, _Real, 3,
Table[Flatten[b, 1], 5000],
CompilationTarget -> "WVM",
RuntimeAttributes -> Listable,
Parallelization -> True
];
Running the actual code:
c2 = Plus[cf[a], cg[b]]; // RepeatedTiming // First
Max[Abs[c1 - c2]]
0.19
0.
Final remarks
The general case may be treated by a suitable combination of ArrayReshape
, MapThread
, Outer
, and Flatten
. Or, maybe even better, by ad-hoc compliled, Listable
CompiledFunction
s such as cf
and cg
instead of MapThread
. Anyways, one would probably need a thourough case analysis for that.
edited Aug 28 at 9:34
answered Aug 27 at 12:43
Henrik Schumacher
36.7k249103
36.7k249103
add a comment |Â
add a comment |Â
up vote
3
down vote
You can use:
a = RandomReal[0,1, 3,5]
b = RandomReal[0,1, 8,5]
c = Table[a[[i]] + b[[j]], i, Length[a], j, Length[b]]
Dimension[c]
will be 3,8,5
, similar to result.shape
in Python of (3,8,5)
5
In this simplified case, alsoOuter[Plus, a, b, 1]
works (and should be much faster).
â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know aboutOuter
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
add a comment |Â
up vote
3
down vote
You can use:
a = RandomReal[0,1, 3,5]
b = RandomReal[0,1, 8,5]
c = Table[a[[i]] + b[[j]], i, Length[a], j, Length[b]]
Dimension[c]
will be 3,8,5
, similar to result.shape
in Python of (3,8,5)
5
In this simplified case, alsoOuter[Plus, a, b, 1]
works (and should be much faster).
â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know aboutOuter
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
add a comment |Â
up vote
3
down vote
up vote
3
down vote
You can use:
a = RandomReal[0,1, 3,5]
b = RandomReal[0,1, 8,5]
c = Table[a[[i]] + b[[j]], i, Length[a], j, Length[b]]
Dimension[c]
will be 3,8,5
, similar to result.shape
in Python of (3,8,5)
You can use:
a = RandomReal[0,1, 3,5]
b = RandomReal[0,1, 8,5]
c = Table[a[[i]] + b[[j]], i, Length[a], j, Length[b]]
Dimension[c]
will be 3,8,5
, similar to result.shape
in Python of (3,8,5)
answered Aug 27 at 12:31
Lee
40817
40817
5
In this simplified case, alsoOuter[Plus, a, b, 1]
works (and should be much faster).
â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know aboutOuter
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
add a comment |Â
5
In this simplified case, alsoOuter[Plus, a, b, 1]
works (and should be much faster).
â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know aboutOuter
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
5
5
In this simplified case, also
Outer[Plus, a, b, 1]
works (and should be much faster).â Henrik Schumacher
Aug 27 at 12:36
In this simplified case, also
Outer[Plus, a, b, 1]
works (and should be much faster).â Henrik Schumacher
Aug 27 at 12:36
@HenrikSchumacher that is pretty cool! I did not know about
Outer
â Lee
Aug 27 at 12:41
@HenrikSchumacher that is pretty cool! I did not know about
Outer
â Lee
Aug 27 at 12:41
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
Yes, I've tried that but it's not fast for large arrays
â Maria Sargsyan
Aug 27 at 14:30
add a comment |Â
1
What is the output of this addition?
â Henrik Schumacher
Aug 27 at 12:16
The output shape is (3,8,5)
â Maria Sargsyan
Aug 27 at 12:17
I don't know of such a built-in method.
â Henrik Schumacher
Aug 27 at 12:28
2
Maybe one can play around with
Outer
.â Henrik Schumacher
Aug 27 at 12:37
1
Outer[Plus, a[[All, 1]], b[[1]], 1]
should be fast.broadcastedJIT[Plus, a, b]
, from my answer, is two times faster on my computer.â jkuczm
Aug 28 at 10:08