Biotissue Weekly Progress

From ScorecWiki

(Difference between revisions)
Jump to: navigation, search
Revision as of 14:11, 6 March 2014
Fovard (Talk | contribs)

← Previous diff
Revision as of 14:12, 6 March 2014
Fovard (Talk | contribs)

Next diff →
Line 15: Line 15:
Dan: Dan:
* Working on adding fiber matrix parallel model. * Working on adding fiber matrix parallel model.
 +* Implementing Neumann BCs.
==Week of 23 Feb 2014== ==Week of 23 Feb 2014==

Revision as of 14:12, 6 March 2014

Back to Biotissue Project


Week of 3 March 2014

Thursday 6


  • Wiki and svn are now public so we can share with UMN.

Wednesday 5

Bill and Dan:


  • Working on adding fiber matrix parallel model.


  • Implementing Neumann BCs.

Week of 23 Feb 2014

Tuesday 25


  • Finished implementing first version of microscale choice i.e. an element decides whether or not to compute a microscale.
  • Bill had some of the functionality there already, I mostly edited the multiscale integrator atPoint function to include two functions, one that uses microscale stresses and one that uses stresses computed from macroscale displacements and linear elasticity.
  • Seems to work fine.
  • Added output to visualize which elements have a microscale.
  • An element must either have or not have microscale for the entire duration of simulation. Making this dynamic (i.e. choosing after every load step which elements should compute a microscale) I believe is more complicated.

Week of 17 Feb 2014

Friday 21


  • CCI changed the file system on the Q (AMOS). The biotissue build for the Q has now been updated to reflect that change. All files and dependencies have been rechecked-out/recreated/rebuilt on the new file system.
  • We should be to the point on the Q that all the legacy file system stuff can be deleted. But of course I'll hold off on that.
  • We may need more space in the "barn" portion of the file system.
  • Now the MACRO executable is placed in scratch space when built. Copies of the run scripts and run data are now kept in scratch space as well as this space should have better performance. Files in scratch are deleted after 56 days.
  • Started a wiki page on general build info: Biotissue Build Information.

Monday 17


  • Over the last week got some preliminary strong and weak scaling results on Q.
  • There are some errors occurring on the Q (petsc seeing a possible divide by zero and memory issues) somewhat randomly, but especially for larger meshes and larger numbers of processes on macroscale.

Week of 10 Feb 2014

Thursday 13


  • More tests on Q with larger meshes.
  • Made some very large meshes with a variety of partitions in preparation for Q runs.

Tuesday 11


  • Q version of code is producing correct results on larger mesh (~5000 elements).
  • Attempting simple strong scaling test on Q over # of microscale processes with 1 macroscale process.

Monday 10


  • Optimized version of code working on Q.
  • Updated corresponding build scripts and such on svn.

Week of 3 Feb 2014

Thursday 6


  • We have multiscale convergence now. Microscale was receiving incorrect data.
  • Simple cube executes in parallel (one of the processes converges), with unverified results.
  • Dog bone gets stuck at LinearSystem_GetVectorNorm when running in parallel.
  • Todo: Parallel calculation of norm for convergence.

Wednesday 5


  • Updated mesh partitioning code to make it more general. Uploaded more complicated meshes for dog bone geometry with 1, 2, and 4 parts.

Tuesday 4


  • Code to create partitioned meshes has been placed under macro/test/MeshUtil , to do: make that code not hard-coded. To view partitioned mesh on scorec system load module simmetrix/simModeler , run meshviewer , then open partitioned mesh and select the directory.

Week of 26 Jan 2014

Monday 27


  • Working on the AMSI developments desired for use in the SIAM PP14 conference. Implementing various functions at all levels of the AMSI infrastructure to allow the adding and removal of process from tasks in the AMSI meta description of the parallel space and working out when best to echo these changes back to the MPI environment for actual effective reconfiguration. Enriching some of the AMSI data structures to support automatic redistribution of parallely-distributed quantities during process reassignment. Developing support for AMSI reconfiguration protocols such as only sending data characterizing the reconfigurations needed when the communication about to take place will be effected by the reconfigurations.

Week of 19 Jan 2014

Tuesday 21


  • Working mesh output using apf function to write vtk files. Mesh elements are tagged prior to output with stress values from microscale RVE computation. Output files can be viewed in paraview. This function requires building with PCU and calling PCU_Comm_switch within the macro side.

Week of 12 Jan 2014

Friday 17


  • Started fleshing out psuedo-code and test-case(s) for AMSI process task reassignment functionality... it's fairly complicated and some changes will likely need to be made and evaluated during implementation, but it is a necessary step in development.
    • Any notes generated as part of this process will be uploaded to the wiki in the next few days as they are fleshed out and come closer to being finalized.

Thursday 16


  • Started developing concepts for AMSI: overlapping task sets and process reassignment for better load balancing and to prevent CPU idle time.
    • Decided that overlapping task sets seem unnecessary if process reassignment is made robust and/or simple enough since overlapping sets would only serve as essentially another step in the process reassignment function -- and limit the function.

Wednesday 15


  • Displacement results from Simple Cube test with a finer mesh look reasonable. Same convergence issue.
  • Getting nans with DogBone test.
    • Update: Dog bone test with course mesh results in reasonable looking displacement results. Same convergence issue as simple cube.
    • Update: Nans were resulting from uninitialized matrix in multiscale integrator.


  • Additional work on AMSI presentation plans for SIAM PP 14, mostly identifying what functionalities we should try to get working in AMSI prior to the presentation.

Tuesday 14


  • Load stepping support has been added for large deformation cases.
    • Makes use of time-dependent boundary conditions specified in simModeler.
    • Introduces a variable "simulation_time" in TMSA class. Currently this variable runs from 0 to 1, corresponding to the initial undeformed state (0) and the final configuration (1). e.g. if the user requests 4 load steps solves will occur at simulation_time = 0.25, 0.5, 0.75, and 1.0.
    • After a load step is complete, the solver sets the BCs again using the same function to initially set them, but with the simulation_time updated. (Note: the BC function no longer sets mesh values, as this was tricky with non-initial BCs. The mesh doesn't appear to be needed for the solve anyways, and is updated afterwards.)
    • The main loop for the analysis has been reduced to one loop (instead of two nested loops) to match the microscale side. Temporary choice, might make more sense to change microscale side, easy change either way.
  • Notes on convergence
    • As stated earlier with the default input (initial displacement of 0.001 on one face of simple cube) convergence hangs around 10e-8. If the initial displacement is reduced to 0.00002, then convergence hangs around 10e-10, implying that a single precision issue is probably not the problem.
    • Using the load stepping does not appear to help the issue. In other words, convergence stops at a certain number for a certain displacement regardless of the number of load steps taken to reach that displacement.


  • Small updates made to accommodate new apf interface specification.
  • Added residual norm output at the end of each nonlinear iteration to speed debugging of that issue.
  • Looked through the code searching for possible sources of single-precision floating point limits... don't appear to be any, and as noted above it seems like this might not be the issue.
  • Finished a few small refactor changes to micro_fm, which is being set aside until after the SiamPP14 conference next month.
  • Added support for the catan workstation to the parallel configuration file for defining MPI environment information.
  • Created the AMSI Talk for Siam PP 14 page and began laying out basic thoughts and overview of the presentation.

Monday 13

Bill: Refactoring work on micro_fm, see here for notes concerning current refactoring of the Biotissue project as a whole.

Pre-Meeting Notes:

  • We're getting convergence down to 10e-8, which is not the same as was being seen on the previous project.. we assume. Neither of us was actually around for a previous working version so we cannot be sure.
    • The floating point literals defining epsilon for convergence have been carried over from the old project into the new project and are around 10e-12. The machine epsilon the machines we're using is float: 10e-7 and double: 10e-16, and we're only using doubles for calculation, so it is doubtful this is a source of error.
    • Picu has said several times that the micro_fo code is inherently unstable and it may be that we're seeing the results of this.
  • If this level of convergence is acceptable, then there are several avenues for immediate work
    • Load stepping should have just been implemented by Dan, we would like to test reducing the immediately applied load for our simplest test case to see if the issue with better convergence is due the initial displacement in the SimpleCube test case being too much for the nonlinear method's we're using to handle (unlikely but possible).
    • We have not yet implemented any variable recovery at the end of processing, only the displacement field values (no stresses), implementing this a combining it with a mesh writing function to output the results needs to happen soon.
    • Currently the only problem being used is the SimpleCube test case which is simple and uninteresting, we should move up to the DogBone test case as soon as possible, in order to work out any conversion issues to new problems, and then work on getting larger meshes running so we can get interesting multiscale results.

Post-Meeting Notes:

  • Most of the issues brought up in the meeting are issues which Dan is going to begin tackling over the next few days or so.
  • Bill is going to start working on the presentation for SIAM PP next month, so that areas can be identified in which work will have the most immediate impact with regard to closing up holes in the presentation, he is also going to be working on the mechanics side of the code enough that anything Dan is working on does not get held up.
Personal tools