Lines of Code
[root]/StandAlone/tools/puda
| Author | Changes | Lines of Code | Lines per Change |
|---|---|---|---|
| Totals | 109 (100.0%) | 709 (100.0%) | 6.5 |
| harman | 29 (26.6%) | 440 (62.1%) | 15.1 |
| beckvermit | 12 (11.0%) | 185 (26.1%) | 15.4 |
| dav | 21 (19.3%) | 54 (7.6%) | 2.5 |
| jas | 46 (42.2%) | 29 (4.1%) | 0.6 |
| ahumphrey | 1 (0.9%) | 1 (0.1%) | 1.0 |
Reverting code to r52880 before my memory use fix commit. While most tests passed, the ones using AMR and the dynamic load balancer were not passing and the weeds are too deep within them to quickly figure out what is causing the problems. Next week I will attempt to figure out what is going on with them. My guess is that it will take quite a bit of effort to understand the inner workings of the dynamic loadbalancer in order to get everything to work, so it may be quite a while before this fix may go in.
10 lines of code changed in 12 files:
Fix so puda will run - some libs now use proc0cout which requires Uintah::Parallel to be initialized (even if we are not running MPI).
5 lines of code changed in 1 file:
This is the first commit (of what may be several over the next month or so) dealing with Uintah's excessive use
of memory with respect to its XML data structures (that are used for I/O) when running on thousands of patches.
This commit specificly addresses the excessive use of memory when restarting a simulation. Before this commit,
Uintah would read in a timestep.xml file (from the checkpoints/t00000/) directory and create an huge DOM
structure in memory. (On 8K patches on Vulcan (BGQ - LLNL) memory per process would increase by 100% (by
20 MB per process on the advect sample problem). On 30K patches, memory use of 200 MB or more has been observed.)
To solve this problem, instead of reading the XML into memory, we parse it on the fly and extract the data we need,
but do not save it. Note, there are two parts of the timestep.xml file that cause the memory issue: the 1000s of
<Grid> and <DataItem> tags. Small XML nodes are not a problem, and thus we still use them for some things (for
example, the component section read from the timestep.xml file. A full DOM of this node is created (during the
streaming read of the rest of the file) and this node is passed down to the component as before.
A number of changes in this commit are cosmetic, dealing with white space and coding standards, and were made
while I was digging through the code to understand where the memory issue was and how to fix it. To better
see real changes in the code, I highly suggest that you use a diff that ignores generic white space changes,
such as:
svn diff --diff-cmd diff --extensions -w <file_name>
Note,asIgetolderInolongercanreadtextlikethis,soIamaddingspacesintothecodetomakeit easier to see. :)
M CCA/Components/SimulationController/AMRSimulationController.cc
- Number of levels in a Grid (grid->numLevels()) cannot be negative and thus should be an unsigned int... it is now.
- Costmetic: English sentences start with capital letters and end with punctuation... yeah yeah I know.
M CCA/Components/SimulationController/SimulationController.cc
- queryGrid: pass through a pointer, don't create a new copy.
- Unsigned int for grid level number.
- Cosmetic white space/coding standards/English updates. Fixed some strange indentation. Untabify.
M CCA/Components/Parent/Switcher.cc
- Unsigned int for grid level number.
- Cosmetic: Swatted some evil endl's. :) (Why write 8 more characters of code, when 2 will do? And makes it easier to add to strings later...)
M CCA/Components/LoadBalancers/ParticleLoadBalancer.cc
M CCA/Components/LoadBalancers/ParticleLoadBalancer.h
- Updates to consolidate code in load balancers... Still needs some direct attention as opposed to this
first pass while working on something else.
M CCA/Components/LoadBalancers/SingleProcessorLoadBalancer.h
- Add 'do-nothing' implementation of restartInitialize() to this class.
M CCA/Components/LoadBalancers/CostModelForecaster.cc
- Cosmetic: Untabbify, white space, {} around one line ifs, English.
M CCA/Components/LoadBalancers/DynamicLoadBalancer.h
M CCA/Components/LoadBalancers/DynamicLoadBalancer.cc
- Moved restartInitialize() (and a number of supporting class d_ variables) into LoadBalancerCommon.
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English.
M CCA/Components/LoadBalancers/ProfileDriver.cc
M CCA/Components/LoadBalancers/CostModeler.h
- grid->numLevels() is now unsigned updates.
- Coding standard cleanups / English / white space.
M CCA/Components/LoadBalancers/LoadBalancerCommon.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
- Make global DebugStream variables be less global (made static to file).
- Unsigned int for grid level number.
- Move restartInitialize() from DynamicLoadBalancer and ParticleLoadBalancer (code was exact duplicate in both files) here.
M CCA/Components/Solvers/AMR/AMRSolver.cc
- Cosmetic: White space, line up variables, English.
- Unsigned int for grid level number.
M CCA/Components/DataArchiver/DataArchiver.cc
M CCA/Components/DataArchiver/DataArchiver.h
- Comment added for where next XML memory fix should take place.
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M CCA/Ports/ProblemSpecInterface.h
M CCA/Components/ProblemSpecification/ProblemSpecReader.h
M CCA/Components/ProblemSpecification/ProblemSpecReader.cc
- Created new version of readInputFile() which takes a list of patches that are specific to this processor. However,
I don't think anyone (me?) is using it yet, and I'm actually not convinced that it is the right function to fix
(see comments in code). I'm committing it just in case it turns out that we need it as we fix the rest of the
XML memory bloat issues. However, at that point if it turns out not to be useful, it should be removed.
M CCA/Components/Schedulers/TaskGraph.h
M CCA/Components/Schedulers/TaskGraph.cc
- Add some consistency to the member variable naming in this file. Half the variables use d_, and the other half
have a trailing _, and the third half ;) didn't do either one... sigh. Some examples.
lb -> d_lb
dts_ -> d_dts
- A few cosmetic fixes - variable line up, white space, etc.
- Removed var/matl from lb->getOldProcessorAssignment()... need to verify that this is correct... see above.
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc
- Removed unused parameters to lb->getOldProcessorAssignment(). However, the DynamicLoadBalancer still uses the
version with parameters (though it doesn't really need them either). It is unclear why getOldProcessorAssignment()
is not declared in LoadBalancerCommon common or several of the other Load Balancers, and also unclear which function
is actually being called. This is defined in the LoadBalancer.h in CCA/Ports/. This needs more investigation.
M CCA/Components/Schedulers/DetailedTasks.h
M CCA/Components/Schedulers/DetailedTasks.cc
- Almost all cosmetic, white space, untabbify, lining up vars, English, etc changes.
- Add trailing _ to numPendingInternalDependencies to denote member var.
M CCA/Components/Schedulers/MPIScheduler.cc
- Few cosmetic cleanups.
M CCA/Components/Schedulers/SchedulerCommon.cc
M StandAlone/tools/pfs/rawToUniqueGrains.cc
M StandAlone/tools/puda/PIC.cc
M StandAlone/tools/extractors/extractV.cc
M StandAlone/tools/extractors/extractF.cc
M StandAlone/tools/extractors/partextract.cc
M StandAlone/partvarRange.cc
M StandAlone/tools/puda/jim1.cc
M StandAlone/tools/puda/jim3.cc
M StandAlone/tools/puda/varsummary.cc
M StandAlone/tools/puda/puda.cc
M StandAlone/tools/puda/POL.cc
M StandAlone/slb.cc
- Cosmetic white space fixes.
- numLevels() casts to avoid warnings.
M CCA/Components/Arches/Radiation/RadHypreSolver.cc
- Few cosmetic changes.
M CCA/Components/Regridder/RegridderCommon.cc
M CCA/Components/Regridder/SingleLevelRegridder.cc
M CCA/Components/Regridder/TiledRegridder.cc
- Cosmetic white space fixes.
- numLevels() casts to avoid warnings.
M CCA/Ports/Output.h
- Use const vars when possible.
- White space, English.
M CCA/Ports/LoadBalancer.h
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/OS/Dir.h
M Core/OS/Dir.cc
M Core/OS/Dir_boost.cc
- Use 'const' when variables don't change.
- Cosmetic: line up variables, untabbify, white space, {} around one line ifs, English.
- Don't use "std::" in half the file and not in the other half.
M Core/Grid/Patch.h
M Core/Grid/Patch.cc
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Grid/Grid.h
M Core/Grid/Grid.cc
- Added readLevelsFromFile(), parseLevelFromFile(), and parseGridFromFile() to parse XML without creating a full DOM.
This provides a 50% memory decrease (per process) on Vulcan (LLNL-BGQ) for 8K processors when restarting the
advect problem. At some point in the future it might be nice to replace my (brute force - straightfoward -
assumes that the input xml file is very well behaved) approach with the streaming reader functions from the
lib xml library. This is not done at this point as it is unclear how well that library will fit into our use
case, how much time it will take to convert to, and the fact that we want to get this pushed out to the users asap.
- Cosmetic: Line up variables, untabbify, white space, {} around one line ifs, English, coding standards.
- numLevels is now an unsigned int.
M Core/Grid/Task.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Grid/SimulationState.h
M Core/Grid/SimulationState.cc
- Cosmetic: Alphabetize/sort #includes.
M Core/Grid/Level.h
M Core/Grid/Level.cc
- Returned patch from addPatch() is not used anywhere so remove the return type.
- Cosmetic: line up variables, untabbify, white space, {} around one line ifs, English.
M Core/Geometry/Vector.h
M Core/Geometry/Vector.cc
- Added the fromString() function.
- Cosmetic: Alphabetize/sort #includes.
M Core/Geometry/IntVector.h
M Core/Geometry/IntVector.cc
- Added the fromString() function.
M Core/Containers/HashTable.h
- Moved the call to first() into the constructor... Not quite sure what the reason for this is,
but everywhere a HashTable is created, "first()" is immediately called, so it makes sense
to just do it in once place instead of having to do it each time it is used.
D Core/Containers/StringUtil.cc
D Core/Containers/StringUtil.h
A + Core/Util/StringUtil.h
A + Core/Util/StringUtil.cc
M Core/Grid/SimulationTime.cc
M Core/Tracker/TrackerServer.cc
M Core/Persistent/Pstreams.cc
M Core/Persistent/Persistent.cc
M Core/Util/FileUtils.cc
- Moved StringUtil.* to Core/Util/ to all its use in other locations (higher up the build tree).
M Core/Containers/sub.mk
- Untabbify.
M Core/DataArchive/DataArchive.h
M Core/DataArchive/DataArchive.cc
- Rename variables so they are readable, eg: d_tsurl becomes d_ts_path_and_filename, d_tstimes becomes d_ts_times, etc.
- Change a few more functions to parse xml file data on the fly (instead of parsing DOMs), eg: queryEndiannessAndBits()
- Use NULL instead of 0.
- Pass strings as 'const &' instead of copying them every time.
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
M Core/ProblemSpec/ProblemSpec.h
M Core/ProblemSpec/ProblemSpec.cc
- Added constructor to take in a string buffer to create a new DOM from. (Use: stream parse an xml file and put the
needed pieces into a string, then turn only that information into a DOM.)
- Added routines to help in the stream parsing of an XML file: findBlock()
- checkForInputError() replaced.
- parseIntVector replaced with fromString() in respective files.
M Core/Util/DynamicLoader.cc
- Alphabetize #includes.
M Core/Util/sub.mk
- Alphabetize
A Core/Util/XMLUtils.h
A Core/Util/XMLUtils.cc
- Added new helper functions for parsing xml files on the fly.
M StandAlone/compare_uda.cc
- Cosmetic: Alphabetize/sort #includes, line up variables, untabbify, white space, {} around one line ifs, English.
- numLevels() is unsigned now.
- Bulletproofing - verify input args are actually directories.
M StandAlone/restart_merger.cc
- Minor cosmetic fixes.
M StandAlone/sus.cc
- Bulletproofing - catch case when user accidentally runs sus with a UDA and not a UPS.
10 lines of code changed in 6 files:
Remove REDSTORM artifacts.
1 lines of code changed in 1 file:
OnTheFlyAnalysis/momentumAnalysis.*
- added contributions from viscous fluxes and pressure forces.
- major refactor/rework and simplification.
puda/ICE_momentum.cc
- change file output format
ups_spec
- momentumAnalysis no longer require the user to define label names.
Those are hard coded. If Arches & Wasatch users want to use this they should talk to me,
I'll add varLabels for those components
3 lines of code changed in 2 files:
Global:
changed variable name matl_jim -> matl
Added a puda option to perform a conntrol volume momentum analysis on ICE udas. This needs further
testing.
437 lines of code changed in 27 files:
Fixes for the CUDA linking for non-static builds. For CUDA builds, we
need to link in all libraries for executables (eg: sus, test programs,
etc) because Core_Util has CUDA in it and thus the extra dlink object
file must be linked against anything that uses Core/Util, but the
dlink file also has code from many other libraries (they all have to link
together to get the full dlink file). It might be possible to
separate all the CUDA linking out, but at least for now this will work.
Created ALL_STATIC_PSE_LIBS and ALL_PSE_LIBS variables in Makefile.in
that can be used to link executables.
Placed a list of all cuda files (.o version) in the variable
DLINK_FILES (it is added to in each sub.mk). Then, once all libraries
are built, this list of files is linked with -dlink (previously was
just linking all the archive (.a) files together, but nvcc won't let
you do this with shared (.so) libs.
Updated a number of sub.mk files to specify the complete list of libs
necessary to link and added in the CUDA_LIBRARY.
Alphabetized and removed duplicates in a lot of sub.mk files.
M CCA/Components/Models/ModelFactory.cc
- Don't use the RadiationDriver if radation was not turned on.
M Makefile.in
M CCA/Components/Models/Radiation/RMCRT/sub.mk
M CCA/Components/Models/Radiation/sub.mk
M CCA/Components/MPM/sub.mk
M CCA/Components/Wasatch/Transport/sub.mk
M CCA/Components/Wasatch/sub.mk
M CCA/Components/Wasatch/Expressions/sub.mk
M CCA/Components/Wasatch/Expressions/Turbulence/sub.mk
M CCA/Components/Wasatch/Operators/sub.mk
M CCA/Components/sub.mk
M CCA/Components/Schedulers/sub.mk
M CCA/Components/Arches/sub.mk
M CCA/Components/Examples/sub.mk
M Core/Util/sub.mk
M StandAlone/sub.mk
M StandAlone/tools/mpi_test/sub.mk
M StandAlone/tools/pfs/sub.mk
M StandAlone/tools/fsspeed/sub.mk
M StandAlone/tools/puda/sub.mk
M StandAlone/tools/tracker/sub.mk
M StandAlone/tools/dumpfields/sub.mk
M StandAlone/tools/graphview/sub.mk
M StandAlone/tools/extractors/sub.mk
M StandAlone/tools/sub.mk
M StandAlone/tools/compare_mms/sub.mk
M StandAlone/Benchmarks/sub.mk
M testprograms/CubeRootTest/sub.mk
M testprograms/TestFastMatrix/sub.mk
M testprograms/RegionTest/sub.mk
M testprograms/TestMatrix3/sub.mk
M testprograms/IteratorTest/sub.mk
M testprograms/Regridders/sub.mk
M testprograms/sub.mk
M testprograms/PatchBVH/sub.mk
M testprograms/Malloc/sub.mk
M testprograms/Thread/sub.mk
M testprograms/BNRRegridder/sub.mk
28 lines of code changed in 1 file:
Fixed the puda pressure issues
16 lines of code changed in 3 files:
The pressure addition broke puda. All pressure files were taken out
0 lines of code changed in 3 files:
new puda scripts for finding the max and average pressure
169 lines of code changed in 6 files:
Updates to allow for a (almost) clean configure/build on LLNL's BGQ Vulcan machine.
M configure.ac
M configure
- Various cleanups.
- Added --with-essl so you have to explicitly ask for it (previously configure
tried to find it for you on certain machines).
- Removed stray 'echo'.
- Added test to determine if we are running on a BGQ machine.
- Don't use 'type' to find an executable, use 'which' instead.
'type' doesn't always behave the way one would expect.
- Allow for using the mpigcc compiler directly.
- Fix the specification of FORTRAN_UNDERSCORE for xlf.
- Use a .d for depend files on BGQ machines.
- If compiling on a BGQ, then use -lmass instead of -lm for the math library.
- Removed test for PNG (we don't need/use/support it anymore).
PNG was need for NRRD support at one point.
- Removed test for MPEG (Fairly sure this was only needed for SCIRun).
- Allow for --with-mpi=built-in (Ie, using mpigcc compilers)
We originally didn't like using mpiCC compilers as they never quite
worked with configure. However, it appears that a lot of work as been
done over the years making them work better. Also, at LLNL, there are
so many paths that are needed (and separate libraries) for MPI support
that it is just easier to allow the mpigcc compiler to provide them.
- Update the --with-mpi check to loop over possible libraries instead of
having to hard code every possible test in a separate SCI_TRY_LINK.
- Fix --with-petsc so that if you give it an invalid dir, it let's you know.
- Add magic PETSc compiler flags for BGQ as we are using the gcc compiler
but PETSc was compiled with xlc...
M configVars.mk.in
- Add IS_BGQ flag.
- Remove PNG and MPEG references.
- Hack: Add in PETSc flags for xlc petsc use with gcc.
- Make a few HAVE_lib/lib_INCLUDE/lib_LIBRARY sections be laid out consistently.
- Don't put @flag@ in the INCLUDES variable... make a stand-alone var for them
and use them the way all the other vars are used.
- Move the specification of the CORE_STATIC_LIBS to after the specification of
the F_LIBRARY (fortran library) as they need to reference it.
M aclocal.m4
- Renamed 'debugging' flag to 'verbose'.
- Remove RPATH from the link line when linking statically.
M Makefile.in
- Make 'cleanreally' remove all libs (lib/*.so lib/*.a lib/*.dylib) in
addition to all .o files.
M build_scripts/so_epilogue.mk
- No real changes, just indentation so it is readable...
M StandAlone/sub.mk
- Remove unwanted tabs, add whitespace to make more readable.
M StandAlone/tools/puda/sub.mk
- Add Core/Datatypes to the list of required static libs for linking.
M include/sci_defs/error_testdefs.h.in
- Added in a #if check that allows an individual file to override
the SCI_ASSERTION_LEVEL. This is required (at least for now)
because on Vulcan (BGQ), if assertions are off, then the Datatypes.cc
file causes a run time segfault (in both debug and optimized
builds). However, if assertions are turned on for this one file,
then it runs correctly. Still need to dig into what is going on...
M CCA/Components/Models/Radiation/sub.mk
- Needed INCLUDE flags should be specified for each library.
M CCA/Components/MPM/ConstitutiveModel/fortran/sub.mk
- Some of the fortran flags that are hard coded for this library
are not valid on the BGQ machine.
M CCA/Components/MPMArches/fortran/param4.h
M CCA/Components/Models/Radiation/fortran/m_param4.h
M CCA/Components/Arches/fortran/param4.h
- Remove C style comments as they don't compile using xlf on Vulcan.
M CCA/Components/MPM/ConstitutiveModel/fortran/MIGUtils.cc
- Add some magic macro hackiness to fix symbol names (dealing with
fortran underscores) instead of hard coding them.
M CCA/Components/DataArchiver/DataArchiver.cc
- Fix spacing/whitespace issues.
- Check for BGQ and if so, don't use fdatasync() as it is not implemented.
M Core/OS/Dir_boost.cc
- Newest ? boost uses boost:filesystem3, so we need to check and use the
correct namespace.
M Core/Math/sci_lapack.cc
- Add some magic macro hackiness to fix symbol names (dealing with
fortran underscores) instead of hard coding them.
- Add warning messages if lapacksvd is called but the code path is
not implemented...
M Core/Datatypes/Matrix.h
M Core/Datatypes/SearchGrid.h
M Core/Datatypes/Datatype.h
M Core/Datatypes/Color.h
M Core/Datatypes/TypeName.h
M Core/Datatypes/NrrdData.h
M Core/Datatypes/String.h
M Core/Datatypes/MatrixOperations.h
M Core/Datatypes/DenseColMajMatrix.h
M Core/Datatypes/SparseRowMatrix.h
M Core/Datatypes/PropertyManager.h
M Core/Datatypes/DenseMatrix.cc
M Core/Datatypes/Image.h
M Core/Datatypes/ColumnMatrix.h
M Core/Datatypes/DenseMatrix.h
- share.h has been removed (Windows dll thing)... Therefore remove
(now) meaningly SCISHARE.
M Core/Malloc/malloc.cc
- If DISABLE_SCI_MALLOC is not defined, then we should not be prototyping
malloc(), free(), etc.
1 lines of code changed in 1 file:
Updated copyright information.
29 lines of code changed in 46 files: