The RANCH RenderFarm User Guide
Part II – the RANCH for Maxwell Render
www.ranchcomputing.com
15-03-03 – March 3, 2015
Welcome to the RANCH automated rendering service, the super-powerful - and affordable supercomputer for all your Maxwell projects! This document contains information specific to
the use of Maxwell Render on the RANCH. Before reading it, we highly recommend you read
Part I first - the General Guide - which includes everything not specifically related to Maxwell
Render.
The total time of a project, which determines its cost, includes:
- the distribution of the project across the network
+ the preparation / voxelization phase
+ the render time itself that you specify in the submit project form
+ and the merging of all the individual MXIs produced by the render nodes into the final MXI
Important: MXI merging changes since version 2.7
Maxwell Render 2.7 introduced MXI compression. The size of the MXI file produced by a
project is smaller, so it will take less time to download the final MXI, even though the
compression ratio can vary considerably with each scene, depending on many parameters
(number of channels, definition, number of emitters, use of Multilight, etc.). Compressed
MXIs typically take a bit more time to merge than their non-compressed equivalent in
low/medium definition, but less time in high definition.
The drawback of MXI compression is that it is no longer possible to accurately predict the
merging time of a project based on the size of the MXI, since:
- the compression ratio is never the same from one project to the next.
- compressed MXIs of the same size do not necessarily take the same time to merge.
- the final merged MXI is heavier than individual MXIs.
So a 500 MB compressed MXI file may be equivalent to a 1 GB uncompressed MXI, or a 4
GB uncompressed one... the merging time will be 4 times longer in the second case, even
though the MXI compressed size is about the same.
General rule to keep in mind: a complete merging on the RANCH Runner generally takes
between 2 and 20 minutes, for typical MXI sizes (from a few MBs up to 2 GB).
Tip: to keep the size of the MXI low, do not use Multilight unless you absolutely need it. If
that is the case, avoid Color Multilight as it can produce really _huge_ MXIs.
1
1) Maxwell Render on the RANCH: specific features
In addition to the general features described in Part I of the RUG, the RANCH offers specific
features for unbiased renderers in general, and Maxwell Render in particular:
- Maxwell 3.x and Maxwell 2.x cooperative projects are supported
- Ability to resume projects to improve image quality
- Ability to target a cooperative sampling level (SL)
- You get back the MXIs files and the bitmaps
- Ability to render several cameras within a single project (MultiCam mode)
- A denoising pass is automatically applied to cooperative renders
- The waiting list displays in real time an approximation of the current global Sampling level
(SL) reached while rendering a cooperative project. The first SL display is available 5 to 6
minutes after the beginning of the project.
- Real time image previews let you check your project visually while it is being rendered.
2) Important information about scene preparation
It is VERY important that you read carefully this section before submitting a scene. It will
help you to avoid common mistakes.
·
Please verify before sending the scene that you have saved it with all the correct
parameters. It is your responsibility to provide the scene with every parameter
correctly set. The scene will be rendered exactly as you send it.
·
Before sending your scene to the farm, reload it on your computer and begin a render
from Maxwell Render (NOT from Maxwell Studio) at full resolution for a few
minutes, just to check that everything is all right and that your scene does not cause a
crash or other potential problem. Some errors – ‘triangles without material’ for
instance - are not caught when rendering from MS, but they crash MR.
·
Do not use accentuated / non-alphanumeric characters, spaces, apostrophes and dots in
the filenames of your project (scene, textures, objects, etc.). To avoid any possible
parsing problems on the RANCH Runner, please use only letters and numbers. You
can also use the “_” sign (underscore) to replace spaces.
·
You can use externally referenced .mxs scenes (introduced in Maxwell 2.6) in your
projects if all these scenes have the 'xref_' prefix (e.g. 'xref_MyProduct.mxs'). Only
the main scene file can have a standard name (e.g. 'MyScene.mxs'). All the referenced
.mxs files must have the 'xref_' prefix, or the server won't know which is the main
scene and the project will be rejected or considered as an animation (with as many
frames as there are .mxs files - see page 6).
2
3) Scene preparation for a still image project – Cooperative mode
To prepare your still image projects, you must use RANCHecker, a free standalone
application developped by the RANCH. This easy to use tool gathers all the assets needed by
your scenes and prepares them as verified, ready-to-render .vum project archives.
RANCHecker is also able to benchmark your system and give you useful cost/time
estimations for your projects.
You can download RANCHecker on our RANCH for Maxwell Render web site (PC and Mac
versions are available).
Since version 2.2, RANCHecker is also able to generate a MultiCam project (one project with
several cameras to render). For more details on MultiCam projects, please read Appendix B of
this manual dedicated to the MultiCam mode.
3
4) Resuming a cooperative project!
This exclusive RANCH feature lets you resume a previously rendered cooperative project.
Let’s say you just rendered a project called “MyHouse.vum” for two hours. You download
your files, check the image and the MXI… but wish you had specified a render time of four
hours, because the Sampling Level is not high enough and the image is still too noisy.
Fortunately, the RANCH is able to resume a cooperative render! If you resend a project with
the exact same name (in our example “MyHouse.vum”) and - obviously - the exact same X /
Y resolution, the RANCH will be able to automatically retrieve the data from its projects
cache and resume the render from the state it was in at the end of the last render. Other things
to know about resuming:
-
All the data from a cooperative project is kept in the cache until the temporary ftp
account for the project expires. Thus if you try to resume an old project (more than 5
days old), no data will be available and the project will be considered as a new project.
-
If you want to resume a project, you must send it from the same RANCH user account
as the previous one.
-
Each resumed project, when finished, will itself be considered as a new project by the
RANCH. It will have its own new ftp directory… and you will be able to resume it! It
means that you can improve the original project progressively, through successive
resume projects!
-
When using the resume feature, the e-mail you receive will confirm that the current
project is indeed a resumed project.
-
Do not worry if, looking at the RANCH queue during the second run, you see that the
displayed SL is not higher than the one displayed during the first run: the MXIs of the
first and second run are simply merged at the end to produce the final MXI.
Another good surprise for the end: when resuming a project, the RANCH will only use the
data from its cache, and will not attempt to decompress the .vum file you send. It means that
you don’t need to resend a big project file (the true scene) each time. Just create a small text
file (it must not be empty though, its size must be > 0), rename it to the exact name of the
original project - in our example it would be “MyHouse.vum” - and send it!
If the same name is found in the RANCH cache, only the previous data will be retrieved, so it
does not matter that the .vum file you send is not a “real” .vum file. Of course, if no data is
available in the cache for a project of this name, it will simply be rejected as an invalid .vum
file.
4
5) Scene preparation for an animation project – Frame mode
The preparation is different for an animation. Here are the steps to follow:
1) Create your animation in the host application (3DS Max, C4D, etc.). In the Maxwell
plugin, choose the ‘Write MXS’ option. The Maxwell plugin will then generate as
many MXS files as there are frames in the animation. Copy all these MXS scenes in a
new directory that you will use to store all your project’s files. Let’s call it DIRA in
our example (but you can give it the name you want, with only letters and numbers –
no spaces nor special characters).
2) Now, you need to gather all the external files needed by your project (textures, etc.).
To do this, open one of the generated MXS scenes in Maxwell Studio, and use the
“Pack’n’Go” function to write all the external files found to another directory (NOT
your project directory with all the MXS). Let’s call this second directory DIRB.
3) Copy all the files in DIRB to DIRA, except the lone MXS scene which has been
created in DIRB by the Pack’n’Go function. After this operation, all the necessary
files are in DIRA (all the MXS files + all required external files, textures, etc.).
4) Compress all the files in DIRA in one archive with the .7z format or the .zip format
(see 5-A for details) depending on the compacter you use. Please do not use another
format.
5) Rename the compressed archive you just created with the extension “.vum”. For
instance, if your archive is “MyAnim.zip”, rename it to “MyAnim.vum”. It is essential
for the automated renderfarm to detect the file as a Maxwell project.
The RANCH knows that a project is a still image if it finds only one main MXS scene in the
.vum file. It will then allocate the same scene to all the render nodes in cooperative mode. If it
finds several main MXS scenes in the archive (that is, not 'xref_' files - see page 3), it will
process the project as an animation (one MXS scene / frame per node). So, if you want to
send a cooperative still image project and not an animation, please make sure that there
is only ONE main .MXS file in your .vum archive.
Please also note that you cannot upload a file with a size > 2 GB to the RANCH. If you
send an animation project composed of many .MXS scenes, this limit can be reached. In
that case, you will have to split your big project into several smaller projects, each with a
size < 2 GB.
For 3ds Max, Maya and Cinema 4D users
You can send your 3ds Max / Maya / Cinema 4D + Maxwell animation directly as a 3ds Max,
Maya or C4D project, instead of having to send a large number of big .MXS files. To do this,
please consult our web sites and PDF user guides dedicated to these applications.
5
6) Choosing a strategy for rendering
You have two possibilities: choosing a maximum render time limit (in minutes) or a
maximum Sampling Level (SL), depending on your preferences. If you are on a deadline, you
will probably want to get the best quality image in a given time. For instance if you have
around two hours you can enter “120” in the “Render phase time limit” field and 30 in the
“Sampling Level” field. The render phase will then be interrupted after 2 hours, or after a
sampling level of 30 is reached (which is not likely ).
Another strategy: you want to get to a sampling level of N (for instance N=15), which you
consider necessary and sufficient for your needs. In that case you could enter 120 minutes (the
maximum time authorized) and 15 in the SL field. The render will then stop when the desired
SL is reached. You must always use this strategy when you render an animation, to
ensure that all frames are of the same quality.
If you are not too sure, for instance you would like a SL of 16 but it is not mandatory, and in
any case you do not want to exceed a 1 hour render phase time, you would enter “60” and
16”. In that case the render phase will be stopped as soon as the 60 minutes limit is reached,
OR a SL of 16 is reached.
Please note that the total time of a project, which determines its total cost, includes several
factors (the example below is for a cooperative project):
-
The time needed to send the scene to all the nodes. This is quick, generally less than
one minute with most scenes, and several minutes with complex scenes.
-
The preparation time (the voxelization can take a while on complex scenes).
-
The render time (the longer part obviously, which you specify the length in the Submit
Project form).
-
The time needed to gather and merge all the MXIs into the final MXI. It depends on
the MXI sizes (see next page).
-
The time to create your temporary ftp account and transfert the files (typically less
than one minute).
We recommend that you use RANCHecker to better estimate the total cost of your project.
6
IMPORTANT: MXI complexity and processing time
The final MXI file is obtained by gathering and merging the cooperative MXI files from all
the RANCH Runner nodes. The complexity of a MXI file depends on the resolution of the
project, the use or not of the Multilight option, the number of light emitters and channels. The
gathering time (copying the MXIs from all the nodes across the network) and the merging
time (the fusion of all these MXIs into the final MXI) depend on the complexity of each MXI.
With large MXIs, a huge amount of data must be transferred back and forth across the
network, and the time required for the gathering + merging phase can be quite long. That
being said, the RANCH Runner offers the highest level of merging performance
available today. The merging time on the RANCH Runner typically takes between 2 and 20
minutes depending on the project complexity.
Warning:
Merging has changed since version 2.7 of Maxwell Render. Please make sure you have read
the details on page 1 of this guide.
If you check the “Multilight” option when saving your scene, the MXI sizes will be much
bigger, and processing time will increase accordingly. Please be very careful when using the
Multilight option, and be aware that it can considerably increase the total processing time of
the project, even if the render time itself is small.
If you absolutely need to use the Multilight features of Maxwell Render, try to avoid Color
Multilight, as it can generate huge MXIs that can eat a lot of RAM and take a very long time
to merge.
7
7) Files produced by the render
* If your project is a still image / cooperative render
You will find in the ftp directory of your project:
-
your original .VUM file.
the final MXI merged after the cooperative render.
bitmap layers you specified in your scene (alpha, IDmaterial, IDmesh, etc.)
a final image of your project (with the suffix “_final”).
a “denoised” version of the final image (with the suffix “_denoised”). The final image
file is automatically filtered at the end of the RANCH process in an attempt to reduce
the amount of digital noise in the picture.
Note 1: the effects of the “denoising” are subtle. It will never replace a longer render time,
and it will never transform a supergrainy image in a perfect image , but it can help to
“clean” the final picture by removing some of the noise. The denoising pass is very fast
and done at no extra cost for the customer.
Note 2: on some images the denoiser may fail. In that case the “_denoised” image will not
be present in the directory. As it is an automated process, the denoising phase cannot be as
refined as if you had denoised the image manually, and in some rare circumstances it
won’t work. Please consider this feature as a bonus free of charge, not as a guaranteed
result
* If your project is an animation
You will find in your directory:
-
your original .VUM file.
-
all the frames of your animation, saved in the bitmap format you chose in the scene. If
for any reason the bitmap output format is not identified, the 32-bit TIF format will be
used.
-
bitmap layers you specified in your scene (alpha, IDmaterial, IDmesh, etc.)
8
Appendix A: the real time previews
Cooperative project preview (still image)
The RANCH lets you check visually your cooperative project while it is being rendered, on a
web page specific to your project in progress. To access this page, click on the
button in the waiting/priority list.
Below is an example of a preview image.
© Scene by Felix de Montesquiou
Notes:
- The first preview is available 5 to 6 minutes after the beginning of the project.
- The preview does not show all the subtleties of the final image.
- The preview image has a maximum definition of 1400 pixels (wide or high).
- The preview is annotated (project name, current SL, render time, real image definition).
- The preview is generated with each SL increase.
- You can click on the REFRESH button to update the preview.
9
Animation preview
The RANCH offers you a very handy feature for checking visually your animation project
when it is being rendered. It displays 256-pixel wide thumbnails of every tenth frame
rendered (frames 0, 10, 20, etc.) on a web page specific to your inprogress project. To access
this page, you just have to click on the
button which appears when your
project is being rendered (and of course if there is something to preview: if each frame of your
animation takes 30 minutes to render, obviously there will be nothing to see during the first 30
minutes :)
Below is an example of what you will be able to see when you click on Preview (the preview
image is always around 1900 pixels wide, its height depends on the number of thumbnails).
Notes:
- At the end of the render, the animation preview image is copied in your project's directory.
10
Appendix B: the MultiCam mode
In some projects, you may have to render several views of the same scene. The MultiCam
mode (integrated in RANCHecker >= 2.2) lets you select the cameras you want to render:
and then writes this information in the .vum archive. When the project is processed, all the
cameras you checked will be rendered one after the other, so you won’t need to send as many
projects as you have views to render. If there is only camera in the project, the above
requester won’t appear when you prepare your project.
Things to know about the MultiCam mode:
-
if you check only one camera to render, the X/Y definition used when rendering will
be the one you enter in the Submit Project form on the RANCH web site
if you check at least two cameras to render, the X/Y definition used for a camera when
rendering will be the definition saved for this camera in the .mxs scene
each camera is rendered in its own subdirectory in the project directory
each camera can have a different definition (width/height)
if you cancel your project while it is rendering, the whole project will be canceled, not
only the current camera.
the waiting list will display “CAM 2/4” for instance if the currently rendered camera is
the second camera out of four.
the list of selected cameras is displayed in the e-mail sent when the project starts
the SL reached by each rendered camera is displayed in the end project e-mail
the preview (see appendix A) works with each camera successively
resuming a MultiCam project is not possible, even if only one camera is selected
11