Lines of Code
[root]/CCA/Components/DataArchiver
| Author | Changes | Lines of Code | Lines per Change |
|---|---|---|---|
| Totals | 66 (100.0%) | 1939 (100.0%) | 29.3 |
| allen | 48 (72.7%) | 1149 (59.3%) | 23.9 |
| dav | 12 (18.2%) | 784 (40.4%) | 65.3 |
| jas | 4 (6.1%) | 3 (0.2%) | 0.7 |
| jholmen | 1 (1.5%) | 2 (0.1%) | 2.0 |
| ahumphrey | 1 (1.5%) | 1 (0.1%) | 1.0 |
minor interface change as vars do not need to be passed, also minor name change so to associate it with a recomute
0 lines of code changed in 4 files:
moved tmp material to constructor/destructor
2 lines of code changed in 1 file:
moved tmp material to constructor/destructor
11 lines of code changed in 1 file:
minor formatting, commnets, and cleanup
139 lines of code changed in 1 file:
added support to handle sole vars similar to reduction vars. Both being global vars
134 lines of code changed in 2 files:
added a query for getting the number of processors used
1 lines of code changed in 1 file:
made the scrubbing of saved vars optional
14 lines of code changed in 2 files:
additional comments
3 lines of code changed in 1 file:
added the ability to save the old data warehouse
291 lines of code changed in 2 files:
Complete the removal of RunLengthEncoder.
1 lines of code changed in 1 file:
refined the application reduction var class
1 lines of code changed in 1 file:
clean up to the in situ to use the set{output,checkpoint methods}
19 lines of code changed in 1 file:
added reduction option for on demand output and checkpointing
109 lines of code changed in 2 files:
removed unneeded var
1 lines of code changed in 1 file:
variable and comment clean up to refer to the material manager rather than the state
3 lines of code changed in 1 file:
removal of shared state and introduction of a material manager
9 lines of code changed in 2 files:
added a managing class for application reduction variables
3 lines of code changed in 1 file:
made the recompute and abort reduction vars
2 lines of code changed in 1 file:
change from restart to recompute time step so to differentiate from restarting a time step from a checkpoint
15 lines of code changed in 2 files:
only do the bcast of the wall time if checkpointing based on the wall time
12 lines of code changed in 1 file:
This fixes an issue with (older/non-compliant?) file systems with respect to deleting directories.
For many years We had a 'hack' in place for AIX file systems to use 'stat' instead of the cheaper
"file->d_type" check. It turns out that we should (have been) use the hack anytime that file->d_type
is DT_UNKNOWN, not just under AIX. This should fix the problem that some people were seeing on some
file systems where (checkpoint) diretories were not being deleted when they should have been.
M CCA/Components/DataArchiver/DataArchiver.h
M CCA/Components/DataArchiver/DataArchiver.cc
- Const'ify a number of method params that are not changed in the functions (and thus should be declared const).
- Add some white space for readability.
- Actually print out warning messages when old checkpoint directories fail to be deleted.
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc
M CCA/Components/SimulationController/SimulationController.cc
- Purely cosmetic: Add some white space for readability.
M CCA/Ports/Output.h
- Match updated interface (ie, use of const) for DataArchiver.
- Add some white space for readability.
M Core/OS/Dir.h
M Core/OS/Dir.cc
- Add 'const' due to addition of const above.
- Add some white space for readability.
- Fix the check that we use to determine if a 'file' is a directory (see above).
M Core/OS/Dir_boost.cc
- Add const, white space, and {} for if statement.
53 lines of code changed in 2 files:
Updates toward getting particle data to work with PIDX. This commit is not the complete fix,
but particles aren't being used with PIDX anyway, so it won't affect anything.
M CCA/Components/DataArchiver/DataArchiver.cc
- Add PIDX support for IntVector.
- Fix the long64_type params.
- Initial setting of particle position index, though this will change soon with the updating of PIDX to use multiple particles at some point.
M CCA/Ports/PIDXOutputContext.cc
- Fix name from UPS spec, just use "CHUNKING_ZFP" instead of "PIDX_CHUNKING_ZFP".
- Fix name of ParticleVars and NCVars output files.
M Core/DataArchive/DataArchive.cc
- Fixes for handling particle data in PIDX. Not a complete fix, but a step in the right direction.
Particle RT tests do NOT pass at this point.
17 lines of code changed in 1 file:
Fix bad var name from sed
2 lines of code changed in 1 file:
added some comments for clarity
4 lines of code changed in 2 files:
Updates to support PIDX particle output (based on work last week with
Sid and Steve). The PIDX code is passing most local (non-particle)
tests, and the particle data is being saved to some extent at this
point - but the numbers do not compare exactly and there are a few other
strange things going on. I'm committing this so that Steve and Sid
can take a look.
(Note, there are a number of debug prints in the PIDX sections that
need to be removed when the other issues are resolved.)
Updated files:
M Components/DataArchiver/DataArchiver.cc
M Ports/PIDXOutputContext.cc
M Ports/PIDXOutputContext.h
16 lines of code changed in 1 file:
This commit updates the code to support a newer version of PIDX, including the beginnings
of particle support. This update passes all of the local RT using UDA output, but
fails on a number of (previously passing) PIDX tests. I am committing so that Sid/Steve
can take a look and fix the issue.
Sid, here is the output from (one of) the failing test:
Running test ---methane_fire_8patch--- (mpi 8 proc) at 01:04:31
Command Line: /usr/bin/time -p /usr/installed/mpich/mpich-3.2-gcc4.9.2-install/bin/mpirun -genvlist MALLOC_STATS -np 8 sus -svnStat -svnDiff methane_fire_8patch.ups
Successfully ran to completion
Comparing udas
*** ERROR, test (methane_fire_8patch) failed uda comparison, tolerances exceeded (256)
See /ssd2/home/dav/SVN/Uintah_Baseline/opt-mpich/local_RT/ARCHES-results/methane_fire_8patch/compare_sus_runs.log.txt for more comparison information.
Changes:
--------
M CCA/Components/DataArchiver/DataArchiver.cc
M CCA/Ports/PIDXOutputContext.h
M CCA/Ports/PIDXOutputContext.cc
- Updates for newer version of PIDX with (I think) the beginnings of particle support.
- Allow for better (finer grained) specification of PIDX flags in .ups input file.
M Core/Util/XMLUtils.h
M Core/Util/XMLUtils.cc
M Core/ProblemSpec/ProblemSpec.h
M Core/ProblemSpec/ProblemSpec.cc
- Added ability to check for unsigned ints.
M StandAlone/tools/extractors/particle2tiff.cc
- Fix compiler warning. White space. Use {} on one line 'if' statements.
M StandAlone/tools/pfs/sub.mk
M StandAlone/Benchmarks/sub.mk
- Link in PIDX.
M configure.ac
M configure
- Newest PIDX requires the zfp library.
66 lines of code changed in 1 file:
Fix VarLabel memory leak. (Note, the use of the VarLabel to synchronize the tasks is a hack and hopefully will be addressed at some point.)
11 lines of code changed in 2 files:
This commit brings the PIDX data I/O branch (named: pidx_xml) back into the trunk.
All PIDX data I/O is, by default, turned off. To use PIDX you have to configure
against the PIDX library (either installing manually and pointing at it, or using
the --with-pidx flag (not 100% certain that this works at the moment)), and you must
set <DataArchiver type="PIDX"> (as opposed to the default which is type="UDA") in
the .ups input file.
Other caveats:
- Particles are not (currently?) supported in PIDX so any simulation that has a
particle data type will fail (most will print an error that PIDX does not
support them, but it seems some just die).
- Simulations that have overlapping patches (this may just be AMR type
sims) (eg: multibox_rmcrt_coal_1L) can save (slightly different) results.
In the multibox_rmcrt_coal_1L, we saw (only) a handful of differences like this:
((A 1 vs a 0 in a couple of locations) Also note that the UDA itself compared
successfully, but the checkpoint underneath it did not...)
DIFFERENCE [int 36, 5, 0] Time: 0.0003106658 Level: 0 Patch1: 0 Patch2: 0 Material: 0 Variable: areaFractionFX
multibox_RMCRT_coal_1L.uda/checkpoints goldStandard/.../multibox_RMCRT_coal_1L.uda/checkpoints
1 0
- "#if HAVE_PIDX" is used inconsistently through a number of files. Some PIDX stuff is #if'd out, while other code
is left in and compiles, it is just not used.
- There are a lot of cleanups that can and should take place wrt the PIDX integration.
While the code passes all the local RT tests (with the exception of particle tests),
I don't believe it has been run at large scale (though it should work) on a BGQ or Quartz@llnl.
Summary of changes: (Note, see pidx_xml branch commits if you want more information.)
M CCA/Ports/ApplicationInterface.h
M CCA/Components/Application/ApplicationCommon.h
- Added the ability to record (and retrieve) the number for the last regrid time step.
M CCA/Components/DataArchiver/DataArchiver.h
M CCA/Components/DataArchiver/DataArchiver.cc
- When saving as PIDX, only create/output timestep.xml on the initial (I/O) time step that occurs after the regrid.
- Fixes that allow PIDX to save correctly (and efficiently) in parallel. If you are ever trying
to trace this back, read the comments from the commit logs for the "pidx_xml" branch.
- Remove createPIDX_dirs() as it was commented out and Sid tells me that PIDX creats its own dirs and we don't need this.
- Lot of white space fixes.
M CCA/Components/Examples/RMCRT_Test.cc
M CCA/Components/PostProcessUda/PostProcess.cc
- The DataArchive (DA) queryVariables() function now also returns the number of materials for each variable.
- "else" and "else if" should be on a separate line to make it easier to read.
M CCA/Components/Models/Radiation/RMCRT/RayGPU.cc
- Differentiate the output between the GPU and non-GPU reports so the viewer knows explicitly whether the GPU code was run.
M CCA/Components/Schedulers/OnDemandDataWarehouse.h
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc
- Cosmetics: Line up variables, use more than 80 chars per line, white space.
Add in comments denoting default param values.
M CCA/Components/Schedulers/SchedulerCommon.cc
- Cosmetics.
M CCA/Components/SimulationController/AMRSimulationController.cc
- Cosmetics: Use more white space, 80+ chars per line for better readability, etc.
M CCA/Ports/PIDXOutputContext.h
- const'ify some params.
- checkReturnCode() is a static function so mark it so.
M CCA/Ports/PIDXOutputContext.cc
- White spacee. Code alignment for readability.
M Core/DataArchive/DataArchive.h
M Core/DataArchive/DataArchive.cc
- Add in PIDX functions. Note, queryPIDXSerial() is used (currently) by standalone codes (eg: puda).
Its purpose is to wrap the setupQueryPIDX()/queryPIDX() calls that have been added to
allow PIDX to truly run in parallel. At some point it would be good to update puda/compare_uda/etc
to run in parallel...
- See previous comments from PIDX branch for more details on the PIDX communicators that are necessary
to transfer data as some data does not live on all cores and thus can't use the golbal comm.
- Rename a few functions (queryAndSetParticlePositionName()) to be more accurate as to what they do.
- There are still a number of hacks in this code that need to be addressed (eg: var_materials map).
M Core/Geometry/IntVector.h
- Mostly cosmetics...
White space for readability.
Place 1-line functions in .h on one line to make it easier to see all provided functions.
M Core/Grid/Grid.cc
- White space, variable alignment for readability.
M Core/Grid/Level.h
M Core/Grid/Level.cc
- White space, coding standard (return type on its own line).
- Fix logic error in computeVariableExtents() that did not handle periodic boundaries correctly.
This fix was suggested as probably the correct way to do it by Todd... seems to pass tests and did fix
the issue (PIDX saving a particular something or other) that led me to it.
M Core/Grid/Patch.cc
- Place "else" on its own line for readability.
M Core/Grid/Variables/GridVariable.h
- Added emitPIDX().
- Opening "{" for functions in .h files goes on same line.
M Core/Grid/Variables/GridVariableBase.h
M Core/Grid/Variables/GridVariableBase.cc
- Organize #includes.
- White space, line up vars, etc for better readability.
- Return type on its own line.
M Core/Grid/Variables/ParticleData.h
- #include ParticleSubset.h for particleIndex.
M Core/Grid/Variables/ParticleSubset.h
- Cosmetic white space.
M Core/Grid/Variables/ParticleVariable.h
- White space, variable line up, etc.
- Added emitPIDX() function for particles. Note, particles do not fully work yet (on PIDX side).
And thus, this function hasn't really been tested.
M Core/Grid/Variables/ParticleVariableBase.h
M Core/Grid/Variables/ParticleVariableBase.cc
M Core/Grid/Variables/PerPatchBase.h
M Core/Grid/Variables/PerPatchBase.cc
M Core/Grid/Variables/ReductionVariableBase.h
M Core/Grid/Variables/ReductionVariableBase.cc
- Cosmetic white space. Coding standards. Etc.
M Core/Grid/Variables/Variable.h
M Core/Grid/Variables/Variable.cc
- Organize #includes.
- Use "const" on params that are const.
- Cosmetic white space, etc.
M Core/Grid/Variables/VarnameMatlPatch.h
- Pass non-changing strings by const reference (instead of copying them).
- White space for readability.
M Core/ProblemSpec/ProblemSpec.cc
- Line up vars, white space.
M R_Tester/helpers/compare_dat_files.pl
- Strip "/" from end of uda names.
- Clean up output to make it more readable (Todd I hope this doesn't break some of your scripts...)
M R_Tester/helpers/compare_dats
- White space.
M R_Tester/helpers/runSusTests.py
- Fix typo.
M StandAlone/compare_uda.cc
- White space.
- Line up vars.
- Coding standard.
- Line up case statement for better readability.
- 0 => nullptr
- query() now returns if the var was found (because PIDX has a list of all variables the simulation starts with
and the vars may not actually exist in the UDA.
- var => var_name (as it is actually the name, and not the var).
- Use 80+ columns.
M StandAlone/partvarRange.cco
M StandAlone/selectpart.cc
M StandAlone/tools/compute_Lnorm_udas.cc
M StandAlone/tools/dumpfields/dumpfields.cc
M StandAlone/tools/extractors/extractF.cc
M StandAlone/tools/extractors/extractS.cc
M StandAlone/tools/extractors/extractV.cc
M StandAlone/tools/extractors/faceextract.cc
M StandAlone/tools/extractors/lineextract.cc
M StandAlone/tools/extractors/partextract.cc
M StandAlone/tools/extractors/particle2tiff.cc
M StandAlone/tools/extractors/timeextract.cc
M StandAlone/tools/puda/AA_MMS.cc
M StandAlone/tools/puda/ER_MMS.cc
M StandAlone/tools/puda/GV_MMS.cc
M StandAlone/tools/puda/ICE_momentum.cc
M StandAlone/tools/puda/PIC.cc
M StandAlone/tools/puda/POL.cc
M StandAlone/tools/puda/asci.cc
M StandAlone/tools/puda/jacquie.cc
M StandAlone/tools/puda/jim1.cc
M StandAlone/tools/puda/jim2.cc
M StandAlone/tools/puda/jim3.cc
M StandAlone/tools/puda/pressure.cc
M StandAlone/tools/puda/puda.cc
M VisIt/interfaces/archiveInterface.cc
- queryVariables() now returns number of materials.
M StandAlone/tools/puda/varsummary.h
M StandAlone/tools/puda/varsummary.cc
- Swat some evil endls.
M build_scripts/build_pidx.sh
- FYI... Not sure if changing PIDX_TAG to "master" actually works...
M buildbot_try.sh
- Added a number of comments about use of this script.
- Fixed some typos.
- Spread command to multiple lines for readability.
241 lines of code changed in 2 files:
name cleanup
1 lines of code changed in 1 file:
cleanup of the namespace for debug streams and removed using namespace std
13 lines of code changed in 2 files:
added a componet tag for the in situ vars
5 lines of code changed in 1 file:
reset the output timestep when the interval changes
10 lines of code changed in 1 file:
exposed more of the DataArchiver to the in situ
85 lines of code changed in 2 files:
addition debug stream naming
2 lines of code changed in 1 file:
restructured teh Dout and DebugStream to register the streams to avoid conflicts
0 lines of code changed in 1 file:
removed the DataArchiver from grim path to perdition
11 lines of code changed in 2 files:
updated in-situ check
1 lines of code changed in 1 file:
Mostly cosemetic commit. Added in a number of "const" declarations and cleaned up
the code around them.
M CCA/Components/DataArchiver/DataArchiver.h
M CCA/Components/DataArchiver/DataArchiver.cc
- Mark methods as const if they don't change object data.
- Pass (all) params as constants where appropriate.
- Return const pointers to object data (as appropriate).
- Align params, white space.
- Rename findAllVariableTypes() to the more accurate findAllVariablesWithType().
- The "d_" vars are being phased out, so using direct getMPIRank() call instead.
M CCA/Ports/LoadBalancer.h
- Rename enum to use CAPITAL names for clarity. (The enum itself should probably be a named type.)
This makes grep'ing for them much easier.
- Remove tabs, white space.
M CCA/Components/LoadBalancers/DynamicLoadBalancer.cc
M CCA/Components/LoadBalancers/LoadBalancerCommon.cc
M CCA/Components/LoadBalancers/ParticleLoadBalancer.cc
M CCA/Components/SimulationController/AMRSimulationController.cc
- White space.
- Remove some "d_" usage.
- Use new LoadBalancer enum NAMES.
- Single line "if" statements need to use {}.
378 lines of code changed in 2 files:
* Updated KokkosOpenMPScheduler.cc to use Kokkos::OpenMP::partition_master() for Kokkos-based builds.
* Added -npartitions and -nthreadsperpartition run-time options.
* Set the Kokkos-OpenMP Scheduler as the default scheduler for Kokkos-based builds.
* Added bulletproofing within SchedulerFactory.cc.
* Updated ProcessorGroup.h to use m_comm for -nthreads 1 or -npartitions 1.
* Added MasterLock.h to help avoid mixing synchronization primitives.
* Replaced use of #include <mutex> with #include <Core/Parallel/MasterLock.h>.
* Replaced use of std::mutex with Uintah::MasterLock.
* Cleaned up whitespace.
This commit extends the Kokkos-OpenMP Scheduler to support parallel execution of tasks within an MPI process.
Scheduler logic is based upon that of the Unified Scheduler, where individual threads are used as task runners.
In the context of the Kokkos-OpenMP Scheduler, individual partitions are used as task runners.
Individual partitions execute Kokkos-based tasks in parallel across the threads belonging to that partition and existing serial tasks serially.
Note, this scheduler requires MPI_THREAD_MULTIPLE support to run with multiple partitions per MPI process.
2 lines of code changed in 1 file:
merged initializeOutput for PIDX
231 lines of code changed in 2 files:
(4 more)