Lines of Code
[root]/CCA/Components/Parent
| Author | Changes | Lines of Code | Lines per Change |
|---|---|---|---|
| Totals | 29 (100.0%) | 92 (100.0%) | 3.1 |
| dav | 11 (37.9%) | 81 (88.0%) | 7.3 |
| jas | 15 (51.7%) | 9 (9.8%) | 0.6 |
| harman | 3 (10.3%) | 2 (2.2%) | 0.6 |
Reverting code to r52880 before my memory use fix commit. While most tests passed, the ones using AMR and the dynamic load balancer were not passing and the weeds are too deep within them to quickly figure out what is causing the problems. Next week I will attempt to figure out what is going on with them. My guess is that it will take quite a bit of effort to understand the inner workings of the dynamic loadbalancer in order to get everything to work, so it may be quite a while before this fix may go in.
25 lines of code changed in 2 files:
This is the first commit (of what may be several over the next month or so) dealing with Uintah's excessive use
of memory with respect to its XML data structures (that are used for I/O) when running on thousands of patches.
This commit specificly addresses the excessive use of memory when restarting a simulation. Before this commit,
Uintah would read in a timestep.xml file (from the checkpoints/t00000/) directory and create an huge DOM
structure in memory. (On 8K patches on Vulcan (BGQ - LLNL) memory per process would increase by 100% (by
20 MB per process on the advect sample problem). On 30K patches, memory use of 200 MB or more has been observed.)
To solve this problem, instead of reading the XML into memory, we parse it on the fly and extract the data we need,
but do not save it. Note, there are two parts of the timestep.xml file that cause the memory issue: the 1000s of
<Grid> and <DataItem> tags. Small XML nodes are not a problem, and thus we still use them for some things (for
example, the component section read from the timestep.xml file. A full DOM of this node is created (during the
streaming read of the rest of the file) and this node is passed down to the component as before.
A number of changes in this commit are cosmetic, dealing with white space and coding standards, and were made
while I was digging through the code to understand where the memory issue was and how to fix it. To better
see real changes in the code, I highly suggest that you use a diff that ignores generic white space changes,
such as:
svn diff --diff-cmd diff --extensions -w <file_name>
Note,asIgetolderInolongercanreadtextlikethis,soIamaddingspacesintothecodetomakeit easier to see. :)
M CCA/Components/SimulationController/AMRSimulationController.cc
- Number of levels in a Grid (grid->numLevels()) cannot be negative and thus should be an unsigned int... it is now.
- Costmetic: English sentences start with capital letters and end with punctuation... yeah yeah I know.
M CCA/Components/SimulationController/SimulationController.cc
- queryGrid: pass through a pointer, don't create a new copy.
- Unsigned int for grid level number.
- Cosmetic white space/coding standards/English updates. Fixed some strange indentation. Untabify.
M CCA/Components/Parent/Switcher.cc
- Unsigned int for grid level number.
- Cosmetic: Swatted some evil endl's. :) (Why write 8 more characters of code, when 2 will do? And makes it easier to add to strings later...)
M CCA/Components/LoadBalancers/ParticleLoadBalancer.cc
M CCA/Components/LoadBalancers/ParticleLoadBalancer.h
- Updates to consolidate code in load balancers... Still needs some direct attention as opposed to this
first pass while working on something else.
M CCA/Components/LoadBalancers/SingleProcessorLoadBalancer.h
- Add 'do-nothing' implementation of restartInitialize() to this class.
M CCA/Components/LoadBalancers/CostModelForecaster.cc
- Cosmetic: Untabbify, white space, {} around one line ifs, English.
M CCA/Components/LoadBalancers/DynamicLoadBalancer.h
M CCA/Components/LoadBalancers/DynamicLoadBalancer.cc
- Moved restartInitialize() (and a number of supporting class d_ variables) into LoadBalancerCommon.
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English.
M CCA/Components/LoadBalancers/ProfileDriver.cc
M CCA/Components/LoadBalancers/CostModeler.h
- grid->numLevels() is now unsigned updates.
- Coding standard cleanups / English / white space.
M CCA/Components/LoadBalancers/LoadBalancerCommon.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
- Make global DebugStream variables be less global (made static to file).
- Unsigned int for grid level number.
- Move restartInitialize() from DynamicLoadBalancer and ParticleLoadBalancer (code was exact duplicate in both files) here.
M CCA/Components/Solvers/AMR/AMRSolver.cc
- Cosmetic: White space, line up variables, English.
- Unsigned int for grid level number.
M CCA/Components/DataArchiver/DataArchiver.cc
M CCA/Components/DataArchiver/DataArchiver.h
- Comment added for where next XML memory fix should take place.
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M CCA/Ports/ProblemSpecInterface.h
M CCA/Components/ProblemSpecification/ProblemSpecReader.h
M CCA/Components/ProblemSpecification/ProblemSpecReader.cc
- Created new version of readInputFile() which takes a list of patches that are specific to this processor. However,
I don't think anyone (me?) is using it yet, and I'm actually not convinced that it is the right function to fix
(see comments in code). I'm committing it just in case it turns out that we need it as we fix the rest of the
XML memory bloat issues. However, at that point if it turns out not to be useful, it should be removed.
M CCA/Components/Schedulers/TaskGraph.h
M CCA/Components/Schedulers/TaskGraph.cc
- Add some consistency to the member variable naming in this file. Half the variables use d_, and the other half
have a trailing _, and the third half ;) didn't do either one... sigh. Some examples.
lb -> d_lb
dts_ -> d_dts
- A few cosmetic fixes - variable line up, white space, etc.
- Removed var/matl from lb->getOldProcessorAssignment()... need to verify that this is correct... see above.
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc
- Removed unused parameters to lb->getOldProcessorAssignment(). However, the DynamicLoadBalancer still uses the
version with parameters (though it doesn't really need them either). It is unclear why getOldProcessorAssignment()
is not declared in LoadBalancerCommon common or several of the other Load Balancers, and also unclear which function
is actually being called. This is defined in the LoadBalancer.h in CCA/Ports/. This needs more investigation.
M CCA/Components/Schedulers/DetailedTasks.h
M CCA/Components/Schedulers/DetailedTasks.cc
- Almost all cosmetic, white space, untabbify, lining up vars, English, etc changes.
- Add trailing _ to numPendingInternalDependencies to denote member var.
M CCA/Components/Schedulers/MPIScheduler.cc
- Few cosmetic cleanups.
M CCA/Components/Schedulers/SchedulerCommon.cc
M StandAlone/tools/pfs/rawToUniqueGrains.cc
M StandAlone/tools/puda/PIC.cc
M StandAlone/tools/extractors/extractV.cc
M StandAlone/tools/extractors/extractF.cc
M StandAlone/tools/extractors/partextract.cc
M StandAlone/partvarRange.cc
M StandAlone/tools/puda/jim1.cc
M StandAlone/tools/puda/jim3.cc
M StandAlone/tools/puda/varsummary.cc
M StandAlone/tools/puda/puda.cc
M StandAlone/tools/puda/POL.cc
M StandAlone/slb.cc
- Cosmetic white space fixes.
- numLevels() casts to avoid warnings.
M CCA/Components/Arches/Radiation/RadHypreSolver.cc
- Few cosmetic changes.
M CCA/Components/Regridder/RegridderCommon.cc
M CCA/Components/Regridder/SingleLevelRegridder.cc
M CCA/Components/Regridder/TiledRegridder.cc
- Cosmetic white space fixes.
- numLevels() casts to avoid warnings.
M CCA/Ports/Output.h
- Use const vars when possible.
- White space, English.
M CCA/Ports/LoadBalancer.h
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/OS/Dir.h
M Core/OS/Dir.cc
M Core/OS/Dir_boost.cc
- Use 'const' when variables don't change.
- Cosmetic: line up variables, untabbify, white space, {} around one line ifs, English.
- Don't use "std::" in half the file and not in the other half.
M Core/Grid/Patch.h
M Core/Grid/Patch.cc
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Grid/Grid.h
M Core/Grid/Grid.cc
- Added readLevelsFromFile(), parseLevelFromFile(), and parseGridFromFile() to parse XML without creating a full DOM.
This provides a 50% memory decrease (per process) on Vulcan (LLNL-BGQ) for 8K processors when restarting the
advect problem. At some point in the future it might be nice to replace my (brute force - straightfoward -
assumes that the input xml file is very well behaved) approach with the streaming reader functions from the
lib xml library. This is not done at this point as it is unclear how well that library will fit into our use
case, how much time it will take to convert to, and the fact that we want to get this pushed out to the users asap.
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English, coding standards.
- numLevels is now an unsigned int.
M Core/Grid/Task.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Grid/SimulationState.h
M Core/Grid/SimulationState.cc
- Cosmetic: Alphabetize/sort #includes.
M Core/Grid/Level.h
M Core/Grid/Level.cc
- Returned patch from addPatch() is not used anywhere so remove the return type.
- Cosmetic: line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Geometry/Vector.h
M Core/Geometry/Vector.cc
- Added the fromString() function.
- Cosmetic: Alphabetize/sort #includes.
M Core/Geometry/IntVector.h
M Core/Geometry/IntVector.cc
- Added the fromString() function.
M Core/Containers/HashTable.h
- Moved the call to first() into the constructor... Not quite sure what the reason for this is,
but everywhere a HashTable is created, "first()" is immediately called, so it makes sense
to just do it in once place instead of having to do it each time it is used.
D Core/Containers/StringUtil.cc
D Core/Containers/StringUtil.h
A + Core/Util/StringUtil.h
A + Core/Util/StringUtil.cc
M Core/Grid/SimulationTime.cc
M Core/Tracker/TrackerServer.cc
M Core/Persistent/Pstreams.cc
M Core/Persistent/Persistent.cc
M Core/Util/FileUtils.cc
- Moved StringUtil.* to Core/Util/ to all its use in other locations (higher up the build tree).
M Core/Containers/sub.mk
- Untabbify.
M Core/DataArchive/DataArchive.h
M Core/DataArchive/DataArchive.cc
- Rename variables so they are readable, eg: d_tsurl becomes d_ts_path_and_filename, d_tstimes becomes d_ts_times, etc.
- Change a few more functions to parse xml file data on the fly (instead of parsing DOMs), eg: queryEndiannessAndBits()
- Use NULL instead of 0.
- Pass strings as 'const &' instead of copying them every time.
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/ProblemSpec/ProblemSpec.h
M Core/ProblemSpec/ProblemSpec.cc
- Added constructor to take in a string buffer to create a new DOM from. (Use: stream parse an xml file and put the
needed pieces into a string, then turn only that information into a DOM.)
- Added routines to help in the stream parsing of an XML file: findBlock()
- checkForInputError() replaced.
- parseIntVector replaced with fromString() in respective files.
M Core/Util/DynamicLoader.cc
- Alphabetize #includes.
M Core/Util/sub.mk
- Alphabetize
A Core/Util/XMLUtils.h
A Core/Util/XMLUtils.cc
- Added new helper functions for parsing xml files on the fly.
M StandAlone/compare_uda.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
- numLevels() is unsigned now.
- Bulletproofing - verify input args are actually directories.
M StandAlone/restart_merger.cc
- Minor cosmetic fixes.
M StandAlone/sus.cc
- Bulletproofing - catch case when user accidentally runs sus with a UDA and not a UPS.
26 lines of code changed in 1 file:
M CCA/Components/Parent/ComponentFactory.cc
- Needed to #include Arches.h (if Arches is specified). Apparently it
was previously brought in indirectly through MPMArches. This fixes
builds where only the Arches component is specified to configure.
M Makefile.in
- Fix informational message on location of svn_info.h
0 lines of code changed in 2 files:
If MPM or Arches is turned off, don't include MPMArches.
4 lines of code changed in 1 file:
- Fixes for OP X Yosemite and a few minor other fixes. Tested on old OSX and 3 flavors of Linux including GPUs.
M configure.ac
M configure
- Fix excess spew from clang compiler during configure.
- Check for version 14 of OS x (Yosemite).
- Removed 'atlas' check under Darwin as it was actually a blas check.
- Put in checks for Blas and Lapack for old and new (yosemite) OS X.
M aclocal.m4
- Fix shell script output error when used in configure to show the results of `basename $1`.
- Clang compiler uses both stderr and stdout (sigh), and outputs in a different order then
other compilers (for --version). This commit handles this.
- Quote a few variables so that if they are blank, the shell doesn't give warning messages.
M include/sci_defs/blas_testdefs.h.in
- Fix to handle new/old OS X and other OSes in a cleaner manner.
M CCA/Components/Parent/sub.mk
- Arches now uses Boost (in Arches.h), so Parent needs to know to compile against boost.
M CCA/Components/Arches/Arches.h
- Move #if guards for multiple inclusion to the top of file so that the compiler has to do less work.
- Alphabetize #includes so I can see what is there.
- #include boost/shared_prt as it is used in this file.
M CCA/Components/Arches/Transport/URHS.h
- One of our compilers errored on the excessive use of 'typename', so trimmed back a little.
M CCA/Components/Arches/sub.mk
- Remove tabs, lined up, alphabetized, removed old crud.
M Core/Thread/ThreadLock.h
- Move #if guard for multiple inclusion to the top of file so that the compiler has to do less work.
- Fixed spelling in #if guard: Threadlock updated to ThreadLock (capital L).
- Lined things up a little.
M Core/DataArchive/DataArchive.h
- Cosmetic lineup of variables.
M Core/DataArchive/DataArchive.cc
- Cosmetic white space and coding standards update,
don't use "std::" if using "using namespace std".
2 lines of code changed in 2 files:
(And a few more...) Sub.mks: for macs, need to link in Core/Thread for these libraries. Also alphabetized, removed tabs, and cleaned up.
17 lines of code changed in 1 file:
Fix indentation so code is readable.
3 lines of code changed in 1 file:
The following commits are mostly cosmetic in nature and should not
change any answers.
Lots of white space and indentation fixes. It is so much easier to
read code when variables are lined up, and values assigned to them
are also lined up.
M CCA/Components/Parent/Switcher.cc
- Variable "no_solver_specified" was misleading. It meant "no command
line solver specified". However, this is the default case and thus
this variable is not needed.
M CCA/Components/Wasatch/Expressions/EmbeddedGeometry/EmbeddedGeometryHelper.h
- InvalidState exception moved to Uintah namespace.
M CCA/Components/Solvers/AMR/AMRSolver.h
- Parent scheduleInitialize() is purely virtual so must be implemented
in each solver. This allows the user to more easily see whether the
scheduleInitialize() step actually does anything or not.
M CCA/Components/Solvers/HypreSolver.cc
- 'firstPassThrough_' is a class variable so needs the '_' suffix to
meet coding standard.
M CCA/Components/Solvers/SolverFactory.cc
M CCA/Components/Solvers/DirectSolve.cc
M CCA/Components/Solvers/HypreSolver.h
M CCA/Components/Solvers/SolverFactory.h
M CCA/Components/Solvers/DirectSolve.h
M CCA/Components/Solvers/CGSolver.h
- Indentation.
- Added (empty) scheduleInitialize() to Solvers that
require it to be implemented but do not actually use it.
M CCA/Ports/SolverInterface.h
- Moved destructor/constructor empty implementations into .h file for
easier viewing.
- Indentation / white space.
M CCA/Ports/SolverInterface.cc
- This file is empty now so should most likely be removed soon.
M CCA/Components/Arches/ChemMix/ClassicTableInterface.cc
- Throw exception if gzipInflate fails (previously it tried to just
kept going and the error was thus hard to find).
M CCA/Ports/sub.mk
- Don't need to compile empty file SolverInterface.cc now.
M Core/Exceptions/ErrnoException.h
M Core/Exceptions/ErrnoException.cc
- Change var name from errno_ to err_ as it might be easy to confuse "errno_" with "errno".
M Core/Exceptions/sub.mk
- Alphabetize list of libraries.
M Core/Exceptions/InvalidValue.cc
- Don't need superfluous "using std::cout".
M Core/Exceptions/InvalidState.cc
M Core/Exceptions/InvalidState.h
- Moved from SCIRun namespace to Uintah for two reasons:
1) SCIRun namespace will eventually go away.
2) InvalidState exception is already in Uintah namespace and these should be consistent.
M Core/IO/sub.mk
- IO now relies on Exception as the UintahZlibUtil now throws an exception if an error occurs.
M Core/IO/UintahZlibUtil.h
- White space / indentation.
- Add comment about what gzipInflate returns.
M Core/IO/UintahZlibUtil.cc
- Throw an exception when an error occurs instead of just return 0, as
the 0 is never checked and causes debugging to be more difficult.
M StandAlone/sus.cc
- Indent / white space.
- Print out which solver is being used.
4 lines of code changed in 1 file:
More housekeeping
renamed directory PatchComber -> ReduceUda
2 lines of code changed in 2 files:
Housekeeping:
-Removed PatchCombiner code and cleaned out calls to it.
0 lines of code changed in 1 file:
Updated copyright information.
9 lines of code changed in 15 files: