AMR
(10 files, 2878 lines)
HyprePreconds
(13 files, 1181 lines)
HypreSolvers
(19 files, 1993 lines)
[root]/CCA/Components/Solvers
AMR
(10 files, 2878 lines)
HyprePreconds
(13 files, 1181 lines)
HypreSolvers
(19 files, 1993 lines)
| Author | Changes | Lines of Code | Lines per Change |
|---|---|---|---|
| Totals | 43 (100.0%) | 268 (100.0%) | 6.2 |
| tsaad | 13 (30.2%) | 213 (79.5%) | 16.3 |
| dsunder | 1 (2.3%) | 32 (11.9%) | 32.0 |
| jas | 17 (39.5%) | 11 (4.1%) | 0.6 |
| dav | 6 (14.0%) | 9 (3.4%) | 1.5 |
| ahumphrey | 6 (14.0%) | 3 (1.1%) | 0.5 |
The main changes in this commit only effect a simulation when <outputNthProc> is
set (> 1) in the .ups file.
This commit allows for PIDX output on every process for a time step
output, while using every Nth process (if so specified) for UDA
checkpoint output. Eventually the need for this Nth Proc changing
hack (and Nth proc output itself) should go away when PIDX fully takes
over I/O.
Note, when using "Nth Process Output", both a checkpoint and a time step save
CANNOT occur on the same time step. Therefore the time step output is postponed
for one time step.
On a slightly different note, we use the term proc badly... we should use process,
or perhaps better yet, rank... as proc is usually taken to mean "processor", but
in MPI's case, means process - and the term MPI uses for process is rank.
D CCA/Ports/LoadBalancer.cc
D CCA/Ports/LoadBalancer.h
A + CCA/Ports/LoadBalancerPort.cc
A + CCA/Ports/LoadBalancerPort.h
- Renamed from LoadBalancer to LoadBalancerPort to avoid confusion with the LoadBalancer itself.
- Added "setNthProc()" so that the SimulationController can change the value as needed.
M CCA/Components/DataArchiver/DataArchiver.h
M CCA/Components/DataArchiver/DataArchiver.cc
- Added postponeNextOutputTimestep(). This pushes off the output of a time step by one
time step. (This occurs if using Nth Process Output and a checkpoint and a time step
where to occur on the same time step.)
- Added savingAsPIDX() boolean query function so that the SimulationController
can avoid doing all of this hack when we are not using PIDX.
- Removed "getCurrentTimestep()" and "getCurrentTime()", as, per the documentation on them,
they should not be used anyway; you should get this info from the shared state. Note,
the DataArchiver name (getCurrentTime()) is different from the shared state name
(getElapsedTime()) which was also confusing.
- Pass const string parameters by reference.
- White space cleanup / alignments.
- If the input file has no <save> labels, but specifies an output interval, throw an exception.
- If the 'logname' is not available when running a simulation, put 'unknown' instead of ignoring.
Should probably find a better fix then 'unknown', but that will have to wait for another day.
- Swat some evil endls. :)
- Don't calculate 'currsecs' on every process, just on rank 0.
- Removed all calls to usingMPI() as Alan deprecated it in a recent commit.
- procOnLevel is (I hope) a boolean array... so declare and use it as such.
- Don't name a variable with a single letter. n -> var_cnt
- Did I mention white space cleanups... Please for the sake of older eyes...
M CCA/Components/ICE/ICE.cc
M CCA/Components/Models/FluidsBased/AdiabaticTable.cc
M CCA/Components/Models/FluidsBased/NonAdiabaticTable.cc
M CCA/Components/Models/FluidsBased/NonAdiabaticTable.h
M CCA/Components/Models/FluidsBased/SimpleRxn.cc
M CCA/Components/Models/FluidsBased/SimpleRxn.h
M CCA/Components/Models/Radiation/RMCRT/Radiometer.cc
M CCA/Components/OnTheFlyAnalysis/MinMax.cc
M CCA/Components/OnTheFlyAnalysis/1stLawThermo.cc
M CCA/Components/OnTheFlyAnalysis/containerExtract.cc
M CCA/Components/OnTheFlyAnalysis/lineExtract.cc
M CCA/Components/OnTheFlyAnalysis/momentumAnalysis.cc
M CCA/Components/OnTheFlyAnalysis/particleExtract.cc
M CCA/Components/OnTheFlyAnalysis/planeExtract.cc
M CCA/Components/OnTheFlyAnalysis/statistics.cc
- Use shared state getElapsedTime()
- Naming conventions: sharedState -> d_sharedState (to match other class vars in file).
- White space / alignments / etc.
M CCA/Components/LoadBalancers/CostProfiler.cc
- White space separation of #includes between CCA and Core
M CCA/Components/Arches/PressureSolverV2.cc
M CCA/Components/Arches/Radiation/RadPetscSolver.cc
M CCA/Components/Examples/Poisson2.cc
M CCA/Components/Examples/DOSweep.cc
M CCA/Components/Examples/RMCRT_Test.cc
M CCA/Components/Examples/RegridderTest.cc
M CCA/Components/Examples/SolverTest1.cc
M CCA/Components/FVM/ElectrostaticSolve.cc
M CCA/Components/ICE/impAMRICE.cc
M CCA/Components/ICE/impICE.cc
M CCA/Components/LoadBalancers/CostProfiler.h
M CCA/Components/LoadBalancers/DynamicLoadBalancer.cc
M CCA/Components/LoadBalancers/ParticleLoadBalancer.cc
M CCA/Components/MPM/AMRMPM.cc
M CCA/Components/MPM/ImpMPM.cc
M CCA/Components/MPM/SerialMPM.cc
M CCA/Components/Parent/Switcher.cc
M CCA/Components/ReduceUda/UdaReducer.h
M CCA/Components/ReduceUda/UdaReducer.cc
M CCA/Components/Regridder/RegridderCommon.h
M CCA/Components/Regridder/RegridderCommon.cc
M CCA/Components/Regridder/SingleLevelRegridder.cc
- LoadBalancer -> LoadBalancerPort
- White space cleanup / alignments.
M CCA/Components/LoadBalancers/LoadBalancerCommon.h
M CCA/Components/LoadBalancers/LoadBalancerCommon.cc
- LoadBalancer -> LoadBalancerPort.
- Add setNthRank().
- getOutputProc() -> getOutputRank().
- Whitespace / alignment.
- Fix comment to indicate getOutputProc is returning an MPI rank.
M CCA/Components/LoadBalancers/ProfileDriver.h
M CCA/Components/LoadBalancers/ProfileDriver.cc
M CCA/Components/Regridder/TiledRegridder.cc
M CCA/Components/Schedulers/MPIScheduler.cc
M CCA/Components/Schedulers/SchedulerCommon.h
M CCA/Components/Schedulers/SchedulerCommon.cc
- Cosmetics: Whitespace / alignment / coding standards / alphabetize #includes / etc.
- LoadBalancer -> LoadBalancerPort
M CCA/Components/Schedulers/OnDemandDataWarehouse.h
M CCA/Components/Schedulers/OnDemandDataWarehouse.cc
M CCA/Components/Schedulers/Relocate.h
M CCA/Components/Schedulers/Relocate.cc
- Name class variables according to naming convention. (eg: aborted -> d_aborted).
- Be consistent with naming convention of class variables in same file.
- LoadBalancer -> LoadBalancerPort.
M CCA/Components/Schedulers/TaskGraph.h
M CCA/Components/Schedulers/TaskGraph.cc
- LoadBalancer -> LoadBalancerPort.
- White space. Name variable "subset" more accurately: "handling_rank". I don't know
why it was named "subset" to begin with as it doesn't appear to be a "set" to me...
M CCA/Components/Schedulers/UnifiedScheduler.cc
- White space.
M CCA/Components/SimulationController/AMRSimulationController.cc
- There are a lot of cosmetic cleanups in this file, but the real/main change is:
- Add in hack to handle changing from "nth process output" (for checkpoints) to
"1 process output" (for time steps) when running with PIDX (and nth process is > 1).
- Declare variables (start) where they are used, not at the top of a function.
- Move the terminal output of the 0th timestep out of the main simulation loop.
- White space / alignment / etc.
- LoadBalancer -> LoadBalancerPort.
M CCA/Components/SimulationController/SimulationController.h
M CCA/Components/SimulationController/SimulationController.cc
M CCA/Components/Solvers/AMR/AMRSolver.cc
M CCA/Components/Solvers/AMR/HypreDriver.cc
M CCA/Components/Solvers/AMR/HypreDriverStruct.cc
M CCA/Components/Solvers/CGSolver.cc
M CCA/Components/Solvers/HypreSolver.cc
M CCA/Components/Wasatch/Expressions/Coordinate.cc
M CCA/Components/Wasatch/Expressions/PoissonExpression.cc
M CCA/Components/Wasatch/Expressions/Pressure.cc
M CCA/Components/Wasatch/Expressions/RadiationSource.cc
M CCA/Components/Wasatch/Wasatch.cc
M CCA/Ports/Scheduler.h
M CCA/Ports/sub.mk
M Core/DataArchive/DataArchive.h
M Core/DataArchive/DataArchive.cc
M Core/Grid/Grid.h
M Core/Grid/Grid.cc
M Core/Grid/Level.h
M Core/Grid/Level.cc
M StandAlone/tools/uda2vis/uda2vis.cc
- LoadBalancer -> LoadBalancerPort.
- Cosmetic white space, alpha, etc.
M CCA/Ports/Output.h
- Removed getCurrentTimestep() and getCurrentTime() as the user should get
them from the shared state.
- Added postponeNextOutputTimestep().
- Added savingAsPIDX().
9 lines of code changed in 4 files:
Bullet proofing - must be using MPI if you want the Hypre solver.
0 lines of code changed in 2 files:
Removal of src/Core/Thread and related refactoring throughout the code-base.
This is the first step in a series of infrastrucutre overhauls to modernize Uintah. Though this all passes local RT (both CPU and GPU tests), I expect some fallout we haven't considered and will be standing by to deal with any issues. Once the dust settles, we will move to replacing Core/Malloc with jemalloc.
* We are now using the standard library for all multi-threading needs within the infrastructure, e.g. std::atomic, std::thread, std::mutex, etc.
* The Unified Scheduler is now the only multi-threaded scheduler, e.g. ThreadedMPIScheduler no longer exists (though the source will soon be placed into an attic).
* Threads spawned by the Unified Scheduler are detached by default (not joinable), allowing for easy, clean and independent execution. There are no longer ConditionVariables used to signal worker threads, just a simple enum for thread-state.
* What was Core/Thread/Time.* is now Core/Util/Time.* - a next step will be to migrate all internal timers, etc to use std::chrono.
* NOTE: Though much cleanup has occurred with this commit, there is still significant cleanup and formatting to be done. The scope of this commit neccessitates a more incremental approach.
3 lines of code changed in 6 files:
remove get3DPointer from Array3Data
32 lines of code changed in 1 file:
More potential improvements to the Hypre solver interface. Apparently, there is NO need to reassemble the coefficient matrix, the solution, and the RHS vectors.
According to Hypre timings, this results in a 4-6X speedup on the MATRIX+VECTOR assembly timings (ONLY).
Those may be just a tiny fraction of the overall time spent in Hypre at small scale < 4K cores.
Their effects, however, remains to be seen at large scale as well as with the Arches DO radiation solver.
4 lines of code changed in 3 files:
remove #if 0 from the HypreSolver interface
3 lines of code changed in 1 file:
remove misleading code that doesn't work
0 lines of code changed in 1 file:
add a few more notes to describe the usage of the new Hypre timers
6 lines of code changed in 1 file:
fix, cleanup, and improve hypre timings in the Hypre solver.
The time reported by sus for Hypre is based on times measure on Rank 0 only.
While it very likely that the Hypre timings for all ranks are similar, we should also be able to measure the maximum time taken across all ranks.
To turn this feature on, simply uncomment #define HYPRE_TIMING in HypreSolver.h and rebuild Uintah.
In the absence of this flag, there is NO COST associated with have these hypre timing functions since they will turn to empty #define macros (see hypre/src/_hypre_utilities.h)
92 lines of code changed in 2 files:
One additional improvement to the Hypre solver interface:
Do NOT allocate/destroy grid every timestep unless necessary (timestep = 1, setupfrequency, restart).
Please let me know if you notice any speedups at large scale (32K+ cores).
62 lines of code changed in 1 file:
Fix a bad bug in the SolverParameters base class that essentially set the Matrix setup frequency to 1 - ALWAYS. Unless the user specifies setupFrequency in the input file or the component developer casts the SolverParameters as a HypreSolver2Params, the setupFrequency is always set to 1. This likely impacted all Wasatch results that required a linear solver since Feb 2014. I think the setupFrequency should not be specified by the user from the input file. In addition, if the Matrix stencil is not changing, then there's no need to re-setup the matrix. Instead, I added a new option updateCoefFrequency that allows one to keep the Matrix structure intact but only update the coefficients. Test this via setUpdateCoefFrequency. I also added this option to the Uintah ups spec. This commit should not impact Arches simulations since Arches sets the setupFrequency through the input file. This commit will break many Wasatch RT tests with minor diffs.
25 lines of code changed in 2 files:
Move non-conflicting classes that were in SCIRun namespace to Uintah namespace.
0 lines of code changed in 1 file:
Allow one to specify the solve frequency for HypreSolver.
Simply add: <solveFrequency spec="OPTIONAL INTEGER"/>
to your spec and then use it in your input file.
More options can be added (solve at timestep 0?, disable carry forward of solution variable?...)
21 lines of code changed in 2 files:
Update copyright date to 2016.
11 lines of code changed in 16 files: