Thanks for the reply. I rebuilt with type "Release with Debug info", and run on both 5 and 3 shapes. 

For 3 and even 4 shapes, I don't get any segfault. However, if I remove the first shape in correspondence.xml and use the remaining 4, I got "basic_string::_S_construct null not valid" error at the very beginning. If I removed the second or the last files, the remaining 4 shapes will run without errors. I think that might be a different issue. I previous preprocess step,  I used 5 shapes when running "groom". Can this be the reason?

If I use 5 shapes with the newly built executable with debug info, I got the following trace when I run it in gdb (attached at the end). I've check the 5 distance transform volumes that I generated in previous "groom" step, and the files look good, i.e. not corrupted. The files at at ftp://ftp.sci.utah.edu/upload/shapeworks/. Naming convention: PE105 is the original binary label map, PE105DT is the output of preprocess1, and PE105B is the output of preprocess2. 

Thanks again,

Wei

============== gdb trace for 5 shapes =======

=========== end of gdb trace for 5 shapes ================

On Wed, Jul 2, 2014 at 3:51 PM, Alan Morris <akenmorris@gmail.com> wrote:

Wei,

Could you build ShapeWorksRun with debugging symbols and send us the stacktrace?  Alternatively, if you could point us to the data, that would be great.

Can you reproduce it with a smaller sample of data (e.g. 3 shapes).

Thanks,
Alan

On Tue, Jul 1, 2014 at 4:01 PM, Wei Liu <weiliu620@gmail.com> wrote:

Dear shapeworks users, 

Today  I got a segmentation fault when running ShapeWorksRun on my CT image of 5 volumes. From the screen log, it seems the segfault happened at the "optimization step".  Below I attached the log, the xml configuration file and the files that have been generated before the segfault happens. 

I was monitoring the memory usage and found the allocated memory was within the limit of my physical memory. 

Any insights on this issue? Or any flag to turn on to print more debug information?

Thanks in advance,

Wei

========= ShapeWorksRun log ===================

weiliu@lando:/scratch/experiments/shapeworks> /scratch/packages/shapeworks.git/build/ShapeWorksRun/ShapeWorksRun lung.correspondence.xml 

m_processing_mode = 3

m_number_of_particles = 256

m_optimization_iterations = 200

m_output_points_prefix = lung

m_output_transform_file = output_transform_file

m_domains_per_shape = 1

m_timepts_per_subject = 1

m_starting_regularization = 10

m_ending_regularization = 0.1

m_iterations_per_split = 200

m_relative_weighting = 1

m_norm_penalty_weighting = 0

m_initial_relative_weighting = 0.05

m_initial_norm_penalty_weighting = 0

m_adaptivity_strength = 0

m_attributes_per_domain = 0

m_checkpointing_interval = 20

m_transform_file = 

m_prefix_transform_file = 

m_procrustes_interval = 0

m_recompute_regularization_interval = 1

m_procrustes_scaling = 1

m_adaptivity_mode = 0

m_keep_checkpoints = 0

m_optimization_iterations_completed = 0

Procrustes scaling is on

Mean curvature magnitude = 0.0410644

Std deviation = 0.0386809

Mean curvature magnitude = 0.0483452

Std deviation = 0.0401311

Mean curvature magnitude = 0.0528453

Std deviation = 0.0415891

Mean curvature magnitude = 0.0442168

Std deviation = 0.0440822

Mean curvature magnitude = 0.039068

Std deviation = 0.0368917

------------------------------

*** Initialize Step

------------------------------

Particle count: 2

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 2points

Writing lung.1.wpts  with 2points

Writing lung.2.wpts  with 2points

Writing lung.3.wpts  with 2points

Writing lung.4.wpts  with 2points

Particle count: 4

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 4points

Writing lung.1.wpts  with 4points

Writing lung.2.wpts  with 4points

Writing lung.3.wpts  with 4points

Writing lung.4.wpts  with 4points

Particle count: 8

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 8points

Writing lung.1.wpts  with 8points

Writing lung.2.wpts  with 8points

Writing lung.3.wpts  with 8points

Writing lung.4.wpts  with 8points

Particle count: 16

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 16points

Writing lung.1.wpts  with 16points

Writing lung.2.wpts  with 16points

Writing lung.3.wpts  with 16points

Writing lung.4.wpts  with 16points

Particle count: 32

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 32points

Writing lung.1.wpts  with 32points

Writing lung.2.wpts  with 32points

Writing lung.3.wpts  with 32points

Writing lung.4.wpts  with 32points

Particle count: 64

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 64points

Writing lung.1.wpts  with 64points

Writing lung.2.wpts  with 64points

Writing lung.3.wpts  with 64points

Writing lung.4.wpts  with 64points

Particle count: 128

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 128points

Writing lung.1.wpts  with 128points

Writing lung.2.wpts  with 128points

Writing lung.3.wpts  with 128points

Writing lung.4.wpts  with 128points

Particle count: 256

Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Performing global step

..........Writing lung.0.wpts  with 256points

Writing lung.1.wpts  with 256points

Writing lung.2.wpts  with 256points

Writing lung.3.wpts  with 256points

Writing lung.4.wpts  with 256points

Writing lung.0.wpts  with 256points

Writing lung.1.wpts  with 256points

Writing lung.2.wpts  with 256points

Writing lung.3.wpts  with 256points

Writing lung.4.wpts  with 256points

------------------------------

*** AddAdaptivity Step

------------------------------

------------------------------

*** Optimize Step

------------------------------

Optimizing correspondences.

Performing global step

BeforeIteration counter = 0

0: -1.95399e-12

1: 2233.63

2: 6109.69

3: 9130.2

4: 13024.7

ENERGY = 7.00618         MinimumVariance = 10

..........Performing global step

BeforeIteration counter = 0

0: 9.722e-12

1: 2632.27

2: 7232.2

3: 10060.2

4: 14207.3

ENERGY = 7.10904         MinimumVariance = 7.94328

..........Writing lung.0.wpts  with 256points

Writing lung.1.wpts  with 256points

Writing lung.2.wpts  with 256points

Writing lung.3.wpts  with 256points

Writing lung.4.wpts  with 256points

Performing global step

BeforeIteration counter = 0

0: 1.0556e-11

1: 3094.51

2: 8472.64

3: 11077.3

4: 15777.5

ENERGY = 7.21294         MinimumVariance = 6.30957

..........Performing global step

BeforeIteration counter = 0

0: 2.98161e-12

1: 3625.51

2: 9820.69

3: 12145.6

4: 17588.4

ENERGY = 7.31406         MinimumVariance = 5.01187

..........Writing lung.0.wpts  with 256points

Writing lung.1.wpts  with 256points

Writing lung.2.wpts  with 256points

Writing lung.3.wpts  with 256points

Writing lung.4.wpts  with 256points

Performing global step

BeforeIteration counter = 0

0: 1.29785e-11

1: 4217.28

2: 11188.9

3: 13313.2

4: 19565.2

ENERGY = 7.40989         MinimumVariance = 3.98107

..........Performing global step

BeforeIteration counter = 0

0: 1.87255e-11

1: 4865.53

2: 12485.8

3: 14779.8

4: 21615.6

ENERGY = 7.50115         MinimumVariance = 3.16228

.........Segmentation fault

============ end of ShapeWorksRun log ===========

============== files generated ==================

-rw-r--r-- 1 weiliu users      6825 Jul  1 14:45 lung.0.wpts

-rw-r--r-- 1 weiliu users      6825 Jul  1 14:45 lung.0.lpts

-rw-r--r-- 1 weiliu users      6790 Jul  1 14:45 lung.1.lpts

-rw-r--r-- 1 weiliu users      6790 Jul  1 14:45 lung.1.wpts

-rw-r--r-- 1 weiliu users      6820 Jul  1 14:45 lung.2.wpts

-rw-r--r-- 1 weiliu users      6820 Jul  1 14:45 lung.2.lpts

-rw-r--r-- 1 weiliu users      6807 Jul  1 14:45 lung.3.wpts

-rw-r--r-- 1 weiliu users      6807 Jul  1 14:45 lung.3.lpts

-rw-r--r-- 1 weiliu users       644 Jul  1 14:45 output_transform_file

-rw-r--r-- 1 weiliu users      6812 Jul  1 14:45 lung.4.wpts

-rw-r--r-- 1 weiliu users      6812 Jul  1 14:45 lung.4.lpts

-rw-r--r-- 1 weiliu users      6817 Jul  1 14:45 lung.d0.mean

-rw-r--r-- 1 weiliu users      6843 Jul  1 14:45 lung.d0.-3.0.mode

-rw-r--r-- 1 weiliu users      6829 Jul  1 14:45 lung.d0.-2.0.mode

-rw-r--r-- 1 weiliu users      6818 Jul  1 14:45 lung.d0.-1.0.mode

-rw-r--r-- 1 weiliu users      6809 Jul  1 14:45 lung.d0.2.0.mode

-rw-r--r-- 1 weiliu users      6814 Jul  1 14:45 lung.d0.1.0.mode

-rw-r--r-- 1 weiliu users      6791 Jul  1 14:45 lung.d0.3.0.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6793 Jul  1 14:45 lung.d0.-3.1.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6790 Jul  1 14:45 lung.d0.-2.1.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6805 Jul  1 14:45 lung.d0.-1.1.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6823 Jul  1 14:45 lung.d0.2.1.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6804 Jul  1 14:45 lung.d0.1.1.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6796 Jul  1 14:45 lung.d0.3.1.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6790 Jul  1 14:45 lung.d0.-3.2.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6775 Jul  1 14:45 lung.d0.-2.2.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6795 Jul  1 14:45 lung.d0.-1.2.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6818 Jul  1 14:45 lung.d0.1.2.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6808 Jul  1 14:45 lung.d0.3.2.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6829 Jul  1 14:45 lung.d0.2.2.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6827 Jul  1 14:45 lung.d0.-3.3.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6793 Jul  1 14:45 lung.d0.-2.3.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6801 Jul  1 14:45 lung.d0.-1.3.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6798 Jul  1 14:45 lung.d0.1.3.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6815 Jul  1 14:45 lung.d0.3.3.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6806 Jul  1 14:45 lung.d0.2.3.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6817 Jul  1 14:45 lung.d0.-3.4.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6817 Jul  1 14:45 lung.d0.-2.4.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6816 Jul  1 14:45 lung.d0.1.4.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6817 Jul  1 14:45 lung.d0.-1.4.mode                                                                                                                                                 

-rw-r--r-- 1 weiliu users      6817 Jul  1 14:45 lung.d0.2.4.mode                                                                                                                                                  

-rw-r--r-- 1 weiliu users      6816 Jul  1 14:45 lung.d0.3.4.mode  

=============== configuration .xml file =====================

<?xml version="1.0" ?>

<!-- List of files containing the set of shape surfaces. -->

<inputs>        

PE105B.nii.gz

PE900B.nii.gz

PE908B.nii.gz

PE916B.nii.gz

PE919B.nii.gz

 </inputs>

 

<!-- 

OPTIONALLY we could specify a set of point files to initialize the optimization like this: 

<point_files> 

torusDT.00.lpts 

torusDT.01.lpts 

:

.

torusDT.19.lpts

</point_files>

-->

<!-- If point files are not specified, then the application will initialize particles 

     by splitting until each shape has this total number of particles. -->

<number_of_particles> 256 </number_of_particles> 

<!-- Iterations between splitting during initialization phase. -->

<iterations_per_split> 200 </iterations_per_split> 

<!-- Starting regularization for the entropy-based correspondence optimization. -->

<starting_regularization> 10.0 </starting_regularization> 

<!-- Final regularization for the entropy-based correspondence. -->

<ending_regularization> 0.1 </ending_regularization> 

<!-- Number of iterations for the entropy-based correspondence. -->

<optimization_iterations> 200 </optimization_iterations> 

<!-- Number of iterations between checkpoints (iterations at which results are saved) -->

<checkpointing_interval> 20 </checkpointing_interval> 

<!-- Prefix for the output files. -->

<output_points_prefix> lung </output_points_prefix>