Directory Core/DataArchive/

Total Files:
3
Deleted Files:
0
Lines of Code:
2663

[root]/Core/DataArchive

Lines of Code

Core/DataArchive/ Lines of Code

Developers

Author Changes Lines of Code Lines per Change
Totals 24 (100.0%) 1057 (100.0%) 44.0
dav 3 (12.5%) 957 (90.5%) 319.0
allen 11 (45.8%) 88 (8.3%) 8.0
harman 2 (8.3%) 5 (0.5%) 2.5
jholmen 1 (4.2%) 3 (0.3%) 3.0
jas 4 (16.7%) 3 (0.3%) 0.7
ahumphrey 1 (4.2%) 1 (0.1%) 1.0
derekhar 2 (8.3%) 0 (0.0%) 0.0

Most Recent Commits

derekhar 2018-12-13 19:14 Rev.: 59080

update pidx interface and version number. This allows configuring with PIDX w/o errors.

0 lines of code changed in 2 files:

  • Core/DataArchive: DataArchive.cc (changed)
allen 2018-12-04 15:58 Rev.: 58992

added support to handle sole vars similar to reduction vars. Both being global vars

28 lines of code changed in 3 files:

  • Core/DataArchive: DataArchive.cc (+27 -13), DataArchive.h (+1 -2)
harman 2018-11-01 17:37 Rev.: 58895

Replaced tabs with spaces.

2 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+2 -2)
allen 2018-10-10 10:00 Rev.: 58802

commented out proc0cout to remove MPI depencency and white space clean up

7 lines of code changed in 2 files:

  • Core/DataArchive: DataArchive.cc (+6 -6), DataArchive.h (+1 -1)
allen 2018-10-08 13:19 Rev.: 58755

added a query for getting the number of processors used

32 lines of code changed in 2 files:

  • Core/DataArchive: DataArchive.cc (+30), DataArchive.h (+2)
ahumphrey 2018-09-11 09:39 Rev.: 58609

* Remove RunLengthEncoder and supporting classes, etc. This is no longer used. Removal will help clarity regarding Uintah::Variable functionality and features. This change has neccesitated explicit inclusion of "unistd.h" in a few spots, most specifically - MPM, for getpid() calls.

* Remove build system support for IPM and MAGMA libs, which have never been used. MAGAMA in fact, may ultimately hender Kokkos GPU asynchrony and performance.

* Generate new configure script with enable_option_checking bulletproofing.

* Delete unused header files.

* A few other cosmetic changes while under the hood, formatting, white space, etc.

1 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+1)
allen 2018-08-30 15:16 Rev.: 58569

added reduction option for on demand output and checkpointing

14 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+14 -14)
dav 2018-07-10 17:43 Rev.: 58405


Updates toward getting particle data to work with PIDX. This commit is not the complete fix,
but particles aren't being used with PIDX anyway, so it won't affect anything.

M CCA/Components/DataArchiver/DataArchiver.cc

- Add PIDX support for IntVector.
- Fix the long64_type params.
- Initial setting of particle position index, though this will change soon with the updating of PIDX to use multiple particles at some point.

M CCA/Ports/PIDXOutputContext.cc

- Fix name from UPS spec, just use "CHUNKING_ZFP" instead of "PIDX_CHUNKING_ZFP".
- Fix name of ParticleVars and NCVars output files.

M Core/DataArchive/DataArchive.cc

- Fixes for handling particle data in PIDX. Not a complete fix, but a step in the right direction.
Particle RT tests do NOT pass at this point.


119 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+119 -36)
harman 2018-05-17 17:07 Rev.: 58244

comment out debugging output.

3 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+3 -3)
dav 2018-04-25 15:35 Rev.: 58189


This commit brings the PIDX data I/O branch (named: pidx_xml) back into the trunk.
All PIDX data I/O is, by default, turned off. To use PIDX you have to configure
against the PIDX library (either installing manually and pointing at it, or using
the --with-pidx flag (not 100% certain that this works at the moment)), and you must
set <DataArchiver type="PIDX"> (as opposed to the default which is type="UDA") in
the .ups input file.


Other caveats:

- Particles are not (currently?) supported in PIDX so any simulation that has a
particle data type will fail (most will print an error that PIDX does not
support them, but it seems some just die).

- Simulations that have overlapping patches (this may just be AMR type
sims) (eg: multibox_rmcrt_coal_1L) can save (slightly different) results.
In the multibox_rmcrt_coal_1L, we saw (only) a handful of differences like this:
((A 1 vs a 0 in a couple of locations) Also note that the UDA itself compared
successfully, but the checkpoint underneath it did not...)

DIFFERENCE [int 36, 5, 0] Time: 0.0003106658 Level: 0 Patch1: 0 Patch2: 0 Material: 0 Variable: areaFractionFX
multibox_RMCRT_coal_1L.uda/checkpoints goldStandard/.../multibox_RMCRT_coal_1L.uda/checkpoints
1 0

- "#if HAVE_PIDX" is used inconsistently through a number of files. Some PIDX stuff is #if'd out, while other code
is left in and compiles, it is just not used.

- There are a lot of cleanups that can and should take place wrt the PIDX integration.
While the code passes all the local RT tests (with the exception of particle tests),
I don't believe it has been run at large scale (though it should work) on a BGQ or Quartz@llnl.

Summary of changes: (Note, see pidx_xml branch commits if you want more information.)

M CCA/Ports/ApplicationInterface.h
M CCA/Components/Application/ApplicationCommon.h

- Added the ability to record (and retrieve) the number for the last regrid time step.

M CCA/Components/DataArchiver/DataArchiver.h
M CCA/Components/DataArchiver/DataArchiver.cc

- When saving as PIDX, only create/output timestep.xml on the initial (I/O) time step that occurs after the regrid.
- Fixes that allow PIDX to save correctly (and efficiently) in parallel. If you are ever trying
to trace this back, read the comments from the commit logs for the "pidx_xml" branch.
- Remove createPIDX_dirs() as it was commented out and Sid tells me that PIDX creats its own dirs and we don't need this.
- Lot of white space fixes.

M CCA/Components/Examples/RMCRT_Test.cc
M CCA/Components/PostProcessUda/PostProcess.cc

- The DataArchive (DA) queryVariables() function now also returns the number of materials for each variable.
- "else" and "else if" should be on a separate line to make it easier to read.

M CCA/Components/Models/Radiation/RMCRT/RayGPU.cc

- Differentiate the output between the GPU and non-GPU reports so the viewer knows explicitly whether the GPU code was run.

M CCA/Components/Schedulers/OnDemandDataWarehouse.h
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc

- Cosmetics: Line up variables, use more than 80 chars per line, white space.
Add in comments denoting default param values.

M CCA/Components/Schedulers/SchedulerCommon.cc

- Cosmetics.

M CCA/Components/SimulationController/AMRSimulationController.cc

- Cosmetics: Use more white space, 80+ chars per line for better readability, etc.

M CCA/Ports/PIDXOutputContext.h

- const'ify some params.
- checkReturnCode() is a static function so mark it so.

M CCA/Ports/PIDXOutputContext.cc

- White spacee. Code alignment for readability.

M Core/DataArchive/DataArchive.h
M Core/DataArchive/DataArchive.cc

- Add in PIDX functions. Note, queryPIDXSerial() is used (currently) by standalone codes (eg: puda).
Its purpose is to wrap the setupQueryPIDX()/queryPIDX() calls that have been added to
allow PIDX to truly run in parallel. At some point it would be good to update puda/compare_uda/etc
to run in parallel...
- See previous comments from PIDX branch for more details on the PIDX communicators that are necessary
to transfer data as some data does not live on all cores and thus can't use the golbal comm.

- Rename a few functions (queryAndSetParticlePositionName()) to be more accurate as to what they do.

- There are still a number of hacks in this code that need to be addressed (eg: var_materials map).

M Core/Geometry/IntVector.h

- Mostly cosmetics...
White space for readability.
Place 1-line functions in .h on one line to make it easier to see all provided functions.

M Core/Grid/Grid.cc

- White space, variable alignment for readability.

M Core/Grid/Level.h
M Core/Grid/Level.cc

- White space, coding standard (return type on its own line).
- Fix logic error in computeVariableExtents() that did not handle periodic boundaries correctly.
This fix was suggested as probably the correct way to do it by Todd... seems to pass tests and did fix
the issue (PIDX saving a particular something or other) that led me to it.

M Core/Grid/Patch.cc

- Place "else" on its own line for readability.

M Core/Grid/Variables/GridVariable.h

- Added emitPIDX().
- Opening "{" for functions in .h files goes on same line.

M Core/Grid/Variables/GridVariableBase.h
M Core/Grid/Variables/GridVariableBase.cc

- Organize #includes.
- White space, line up vars, etc for better readability.
- Return type on its own line.

M Core/Grid/Variables/ParticleData.h

- #include ParticleSubset.h for particleIndex.

M Core/Grid/Variables/ParticleSubset.h

- Cosmetic white space.

M Core/Grid/Variables/ParticleVariable.h

- White space, variable line up, etc.
- Added emitPIDX() function for particles. Note, particles do not fully work yet (on PIDX side).
And thus, this function hasn't really been tested.

M Core/Grid/Variables/ParticleVariableBase.h
M Core/Grid/Variables/ParticleVariableBase.cc
M Core/Grid/Variables/PerPatchBase.h
M Core/Grid/Variables/PerPatchBase.cc
M Core/Grid/Variables/ReductionVariableBase.h
M Core/Grid/Variables/ReductionVariableBase.cc

- Cosmetic white space. Coding standards. Etc.

M Core/Grid/Variables/Variable.h
M Core/Grid/Variables/Variable.cc

- Organize #includes.
- Use "const" on params that are const.
- Cosmetic white space, etc.

M Core/Grid/Variables/VarnameMatlPatch.h

- Pass non-changing strings by const reference (instead of copying them).
- White space for readability.

M Core/ProblemSpec/ProblemSpec.cc

- Line up vars, white space.

M R_Tester/helpers/compare_dat_files.pl

- Strip "/" from end of uda names.
- Clean up output to make it more readable (Todd I hope this doesn't break some of your scripts...)

M R_Tester/helpers/compare_dats

- White space.

M R_Tester/helpers/runSusTests.py

- Fix typo.

M StandAlone/compare_uda.cc

- White space.
- Line up vars.
- Coding standard.
- Line up case statement for better readability.
- 0 => nullptr
- query() now returns if the var was found (because PIDX has a list of all variables the simulation starts with
and the vars may not actually exist in the UDA.
- var => var_name (as it is actually the name, and not the var).
- Use 80+ columns.

M StandAlone/partvarRange.cco
M StandAlone/selectpart.cc
M StandAlone/tools/compute_Lnorm_udas.cc
M StandAlone/tools/dumpfields/dumpfields.cc
M StandAlone/tools/extractors/extractF.cc
M StandAlone/tools/extractors/extractS.cc
M StandAlone/tools/extractors/extractV.cc
M StandAlone/tools/extractors/faceextract.cc
M StandAlone/tools/extractors/lineextract.cc
M StandAlone/tools/extractors/partextract.cc
M StandAlone/tools/extractors/particle2tiff.cc
M StandAlone/tools/extractors/timeextract.cc
M StandAlone/tools/puda/AA_MMS.cc
M StandAlone/tools/puda/ER_MMS.cc
M StandAlone/tools/puda/GV_MMS.cc
M StandAlone/tools/puda/ICE_momentum.cc
M StandAlone/tools/puda/PIC.cc
M StandAlone/tools/puda/POL.cc
M StandAlone/tools/puda/asci.cc
M StandAlone/tools/puda/jacquie.cc
M StandAlone/tools/puda/jim1.cc
M StandAlone/tools/puda/jim2.cc
M StandAlone/tools/puda/jim3.cc
M StandAlone/tools/puda/pressure.cc
M StandAlone/tools/puda/puda.cc
M VisIt/interfaces/archiveInterface.cc

- queryVariables() now returns number of materials.

M StandAlone/tools/puda/varsummary.h
M StandAlone/tools/puda/varsummary.cc

- Swat some evil endls.

M build_scripts/build_pidx.sh

- FYI... Not sure if changing PIDX_TAG to "master" actually works...

M buildbot_try.sh

- Added a number of comments about use of this script.
- Fixed some typos.
- Spread command to multiple lines for readability.

838 lines of code changed in 2 files:

  • Core/DataArchive: DataArchive.cc (+759 -254), DataArchive.h (+79 -29)
allen 2018-03-29 12:25 Rev.: 58014

cleanup of the namespace for debug streams and removed using namespace std

6 lines of code changed in 2 files:

  • Core/DataArchive: DataArchive.cc (+1 -2), DataArchive.h (+5 -4)
allen 2018-03-17 13:47 Rev.: 57948

restructured teh Dout and DebugStream to register the streams to avoid conflicts

1 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.cc (+1 -1)
jholmen 2018-01-25 22:25 Rev.: 57731

* Updated KokkosOpenMPScheduler.cc to use Kokkos::OpenMP::partition_master() for Kokkos-based builds.
* Added -npartitions and -nthreadsperpartition run-time options.
* Set the Kokkos-OpenMP Scheduler as the default scheduler for Kokkos-based builds.
* Added bulletproofing within SchedulerFactory.cc.
* Updated ProcessorGroup.h to use m_comm for -nthreads 1 or -npartitions 1.
* Added MasterLock.h to help avoid mixing synchronization primitives.
* Replaced use of #include <mutex> with #include <Core/Parallel/MasterLock.h>.
* Replaced use of std::mutex with Uintah::MasterLock.
* Cleaned up whitespace.

This commit extends the Kokkos-OpenMP Scheduler to support parallel execution of tasks within an MPI process.

Scheduler logic is based upon that of the Unified Scheduler, where individual threads are used as task runners.

In the context of the Kokkos-OpenMP Scheduler, individual partitions are used as task runners.

Individual partitions execute Kokkos-based tasks in parallel across the threads belonging to that partition and existing serial tasks serially.

Note, this scheduler requires MPI_THREAD_MULTIPLE support to run with multiple partitions per MPI process.

3 lines of code changed in 1 file:

  • Core/DataArchive: DataArchive.h (+3 -3)
jas 2018-01-09 17:40 Rev.: 57604

Update copyright date.

3 lines of code changed in 4 files:

  • Core/DataArchive: DataArchive.cc (+1 -1), DataArchive.h (+1 -1), sub.mk (new)
Generated by StatSVN 0.7.0