<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Apple Color Emoji";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi Thierry,<o:p></o:p></p>
<p class="MsoNormal"> I took a quick look through your simulation and the wavelength range at the start is quite large especially since you are feeding it though a filter and onto a monochromator. One way to speed things up is to only feed the instrument
the wavelengths you need to go through the Monochromator. I have often tied the wavelength band to be some window around the wavelength set for the Monochromator. Then this window can be adjusted to tradeoff simulation speed for how far from the center wavelength
you want to go.<o:p></o:p></p>
<p class="MsoNormal">If you need to check lambda/n, specific different simulations will give you better results. However if you need all the wavelengths at once you probably need to be running 10^8 neutrons for this wide bandwidth range.<o:p></o:p></p>
<p class="MsoNormal">As you discovered, the MPI feature is the only way to run lots of neutrons. It scales well. So if you can find 32 or 64 cores or better yet 100s or 1000s, that is the best way to go.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If you use split I would recommend using it once. Probably right after the monochromator. However, as Peter said, you have to get enough neutrons to the split to help you. So this may still need 10^8 at the beginning.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best of Luck!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Garrett<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-family:Helvetica;color:black">Garrett E. Granroth<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Helvetica;color:black">Instrument Scientist<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Helvetica;color:black">Neutron Scattering Division<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Helvetica;color:black">Oak Ridge National Laboratory<br>
P.O. Box 2008 MS 6475<br>
Oak Ridge TN 37831<br>
Tel: 865-241-8241<br>
Fax: 865-574-6080<br>
<a href="mailto:granrothge@ornl.gov"><span style="color:#0563C1">email:granrothge@ornl.gov</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">mcstas-users <mcstas-users-bounces@mcstas.org> on behalf of Peter Kjær Willendrup <pkwi@fysik.dtu.dk><br>
<b>Date: </b>Thursday, June 18, 2020 at 2:08 PM<br>
<b>To: </b>Thierry Bigault <bigault@ill.fr>, McStas <mcstas-users@mcstas.org><br>
<b>Subject: </b>[EXTERNAL] Re: [mcstas-users] Inconsistent results using MPI and SPLIT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Dear Thierry,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Just a quick response for now, will have a closer look later.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">One has to be quite careful with SPLIT, it is bossing, and if the statistics is not “big enough” it will most certainly lead to bogus and unstable results. What big enough means is hard to quantify, but generally speaking one typically
needs on the order of 1e5-1e6 events at the point of split at the given ncount.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I didn’t check your instrument file, but one very important detail is that SPLITS must be inserted right before components that apply Monte Carlo / random numbers, otherwise one event will simply become 10 identical and thus drive the errorbars
to unphysically low values. And 5 splits (each 10) is 10^5 and sounds like a quite aggressive splitting.
<span style="font-family:"Apple Color Emoji"">😊</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I am sure we will find a good way to fix this, but I don’t have time this side of the weekend.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Peter<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Hent <a href="https://aka.ms/o0ukef">Outlook til iOS</a><o:p></o:p></p>
</div>
</div>
<div id="id-8b3d29c7-938b-4a46-827c-1f9260a485fb">
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" noshade="" style="color:black" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b>Fra:</b> mcstas-users <mcstas-users-bounces@mcstas.org> på vegne af Thierry Bigault <bigault@ill.fr><br>
<b>Sendt:</b> torsdag, juni 18, 2020 7:03 PM<br>
<b>Til:</b> McStas<br>
<b>Emne:</b> [mcstas-users] Inconsistent results using MPI and SPLIT <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Dear All, <br>
<br>
I built the attached instrument file to try and simulate a reflectometer. <br>
When I saw that I needed 10 minutes of calculation to get 3 neutrons on <br>
my sample, I thought it was time to try and optimize. <br>
After having a look at the manual, I first put some SPLIT keywords <br>
positioned at 5 different components, which I estimate as strategic. I <br>
gained a lot in statistics on the sample. <br>
Then I decided to use the MPI feature, as my laptop has 8 cores. It <br>
nicely reduces the computing time, at first sight it looks really great ! <br>
But the results looked strange, so I made some more systematic tests and <br>
the result is on the attached plots. With all SPLIT commented ("no <br>
SPLIT") it looks fine, the calculated intensity on the sample is <br>
consistent within the error-bars, whatever the number of nodes I use. <br>
When combining all SPLITs active and MPI ("5 SPLIT"), the calculation <br>
time and error-bars can be strongly reduced but the result depends <br>
completely on the number of nodes, with differences much larger than the <br>
error-bar... <br>
Either I did something wrong, or there is a bug somewhere. If someone <br>
has an idea about this issue, I would be interested. <br>
I use version 2.6.1 (May 04, 2020) on Windows 10. <br>
<br>
Thierry <br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>