Posts tagged google

Week 4 into GSoC 2024: Weight transfer experiments, hardships, and results!

Well, this week was really intense. I spent most of the time trying to transfer the weights from the pre-trained PyTorch model of the TractoInferno dataset to the Keras model. I must say that thanks to the reduced size of the AutoEncoder, it was feasible to do it layer by layer without going crazy.

Read more ...


Third Week into GSoC 2024: Replicating training parameters, approaching replication

This week was slightly less productive because I was really busy with my PhD tasks, but I managed to progress nevertheless. After implementing custom weight initializers (with He Initialization) for the Dense and Conv1D layers in the AutoEncoder (AE), I launched some experiments to try to replicate the training process of the original model. This yielded better results than last week, this time setting the weight decay, the learning rate, and the latent space dimensionality as shown in the FINTA paper. Now the AE has no problem learning that the bundles have depth, and the number of broken streamlines decreased a lot compared to the previous results. I also worked on trying to monitor the training experiments using TensorBoard, but I did not succeed because it was a last minute idea and I did not have time to implement it properly.

Read more ...


Second Week into GSoC 2024: Refactoring the AutoEncoder, preliminary results

This week I refactored the AutoEncoder code to match the design patterns and the organization of other Deep Learning models in the DIPY repo; and to make the training loop more efficient and easy to use. I transferred my code to a separate repo to keep the DIPY repo clean and to experiment freely. Once the final product is working, I will merge it into DIPY. I also packaged the whole repo so I can use it as a library. Training experiments were run for a maximum of a 150 epochs, with variable results. They are not amazing, but at least we get some reconstruction of the input tracts from FiberCup, which seems to be on the right track. I also implemented training logs that report the parameters I used for training, so I can reproduce the results at any time. This still needs work though, because not all parameters are stored. Need to polish! The left image shows the input tracts, and the middle and right images show two reconstructions from two different training experiments.

Read more ...


My Journey Continues: Week 1 Progress with DIPY

Hello everyone, I am back with another update on my Google Summer of Code (GSoC) journey with DIPY. The Community Bonding period has come to an end, and I am now fully immersed in the Coding phase of the project.

Read more ...


First Week into GSoC 2024: Building the AutoEncoder, writing the training loop

I finished becoming familiar with the TensorFlow + Keras basics and I wrote the training loop and a couple of scripts for instantiating and training the AutoEncoder. Data loading was also addressed and I am able to load the data from the FiberCup dataset in .trk format using NiBabel, transform it into NumPy arrays, and feed it into the network.

Read more ...


My Journey Begins: Community Bonding Period with DIPY

Hello everyone, I am thrilled to share that I have been selected as a Google Summer of Code (GSoC) student for 2024. Over the summer, I will be working with DIPY, and I am incredibly excited about the journey ahead.

Read more ...


Community Bonding Period Summary and first impressions

Hi everyone! I am Iñigo Tellaetxe Elorriaga, BSc in Biomedical Engineering and MSc in Biomedical Technologies in Mondragon Unibertsitatea, Basque Country. I am a first year PhD student in the Computational Neuroimaging Laboratory in the Biobizkaia Health Research Institute, also in the Basque Country. In the lab, our main paradigm is brain connectivity, so I am familiar with diffusion MRI and tractography. My main lines of research are brain aging, age modelling, and neurorehabilitation, all in the presence of neurodegenerative diseases and acute brain injuries. As of my programming skills, I am mainly a Python developer and I am one of the main contributors to the ageml library, which we are developing at our lab as part of my PhD thesis. I also worked in the industry as a research engineer in the field of medical computer vision for Cyber Surgery, developing new methods to generate synthetic CT images from MRI for reducing ionizing radiation in spinal surgery patients, using generative diffusion models. I have been using DIPY for a while now for my research and other projects, so I am obviously really excited to contribute to the project this summer.

Read more ...


Doing Final Touch-Ups: Week 14

This week I fixed the test for the isotropic source of kurtosis so now it’s working for all DTDs. I also created tests for the K_micro function. Initially while running the test, I got some errors which made me look deeper into the actual function. The error was that I was doing sqrt of some elements when actually I was supposed to calculate square of them. Also I was using a ‘1/5’ factor which was not actually required. On fixing these issues, the overall map image of K_micro improved significantly. Previously the multi-voxel test case was failing due to different eigenvalues in the isotropic total diffusion tensors simulations. Removing the eigenvalue assertion made the test pass, as verifying the kt, cvt, and evals values sufficed. I also provided documentation to some functions in the test file such as _perpendicular_directions_temp_ and from_qte_to_cti etc. Also had to change the name of some functions to make it sound more relatable to what they were actually doing. This week I almost finished with the CTI tutorial. The only thing remaining is to create a fetcher for the data so that all the users can download the data and use it. Currently the path given for the data retrieval is of my local system. I also added some references and overall improved the wording and information in the tutorial. I also finished up writing my final work report and get it reviewed by my mentors and then updated it. Finally, before pushing the file onto the main PR, I cleaned up the code by removing all the extra comments and some unnecessary code and overall made sure that the entire code was following the pep8 standard.

Read more ...


Writing Tests & Making Documentation: Week 13

This week, I finished writing tests for the sources of kurtosis. While the isotropic source passed the test for the anisotropic DTD, the anisotropic source passed tests for all DTDs. As a result, I integrated the test for the anisotropic source within the test_cti_fits function, eliminating the need for a separate function. I also created tests for multi-voxel cases but the tests passed only for single voxel cases. One reason I think this might be happening is because of the way we’re accessing the covariance and diffusion tensor elements. I intend to look further into this. I worked on the real life data, trying to plot maps, but it didn’t work out because the problem was really related to the fact that the current kurtosis source implementations do not handle multiple voxel cases. I worked on real life data, attempting to plot maps. Even though I was not able to get the desired result, I’m sure I’ll figure it out with further research and possible collaboration.

Read more ...


Finalized experiments using both datasets: Week 12 & Week13

Monai’s VQVAE results on T1-weighted NFBS dataset, 125 samples, for batch size of 5 were qualitatively and quantitatively superior to all previous results. I continued the same experiments on the T1-weighted CC359(Calgary-Campinas-359) public dataset consisting of 359 anatomical MRI volumes of healthy individuals. Preprocessed the data using existing transform_img function -

Read more ...


Week12: Making Test Functions Work

According to last week’s situation, I was trying to make the test_cti_fits function work as well as the test_split_cti_params function. This week I figured out the problem with these functions and was able to fix it. The major problem was occurring while I was trying to compare the parameters. Firstly, I started by removing all the extra fit methods in common_fit_method list such as NLS, CLS and CWLS as I realized that we won’t immediately be needing an extra multi_tensor_fit function in CTI.

Read more ...


Making the Tests Work : Week 11

Previously, I had the function for different sources of kurtosis outside the Fit class. Upon suggestion from my mentor, this week I put them inside the Fit class. This required me to make changes to how certain variables were being called inside those functions. I also had to determine what arguments needed to be passed to those functions.

Read more ...


Carbonate issues, GPU availability, Tensorflow errors: Week 10 & Week 11

Recently, I’ve been an assigned RP(Research Project) account on University of Bloomington’s HPC cluster - Carbonate. This account lets me access multiple GPUs for my experiments in a dedicated account.

Read more ...


Adding Tests : Week 10

Last week, we decided to generate a DTD to make the model more robust. This decision accounted for situations where almost all the parameters were non-zero. However, the signals weren’t matching exactly in that situation. This week, I fixed that issue. We can now safely say that all DTDs will match the ground truth signals, regardless of which parameters are non-zero or what changes we make. We accomplished this by figuring out the correct order of ccti parameters. These are covariance parameters that take the root2 and ‘2’ factor into consideration.

Read more ...


Generating Fit Functions : Week 8 & 9

This week, I started by figuring out how to run Spyder on Ubuntu. After resolving technical problems, I needed to ensure I could edit code to meet pep8 standards but the automatic formatting of code wasn’t working. I made changes in the utils.py file to increase the design matrix readability and fixed a typo in the B[:,3] and B[:, 4] diffusion tensor elements. This is because we realized that the sign needed to be negative to show that it’s representing a signal decay. I implemented mapping all of the covariance parameters from paper to its actual code, creating a need to talk to the original paper’s authors as the conversion shown in the paper didn’t quite match its implementation. I also worked on matching the ground truth signal values in case of anisotropic and combined DTDs. This is because the isotropic DTD signals that were being generated matched exactly the QTI signals, as in case of isotropic we’ve 6 non zero elements, and the rest are 0s. However in anisotropic case we had more non-zero covariance parameters (9 non-zero), similarly as in the case of combined DTD. So we figured out that the non-zero elements are being multiplied to some value which isn’t correct and that this needs modifying the ccti conversion. So, I worked on reading more about voigt notation, as the QTI parameters were implemented using that notation. Then we looked again into the QTI paper, and felt the need to contact their author and code implementer and realized that the coding was done while keeping in mind the voigt notation conversion as well as some other factors. At the end of this we figured out the correct conversion of the ccti parameters. We noticed that some factors needed the (root2) division, while some others needed (2). Therefore, we were able to successfully figure out the correct factors that needed to be multiplied/ divided to each of the covariance parameters. And hence, now the signal values of all the DTDs match as expected. Then the other major ongoing task this week has been the implementation of the Fit class in CTI. This required me to implement some functions which might’ve been implemented in DKI/ QTI. This is an ongoing task and would require more work.

Read more ...


VQVAE MONAI models & checkerboard artifacts: Week 8 & Week 9

We observed in our previous results that the Diffusion Model’s performance may depend on better and effective latents from VQVAE. After playing around with convolutional & residual components in the existing architecture that yielded unsatisfactory results, we decided to move to a more proven model on 3D MRIs. It is not necessary that a model that worked well on MNIST dataset would also deliver similarly on 3D MRI datasets, owing to the differences in complexity of the data distributions. Changing the convolutions to 3D filters alone clearly did not do the job.

Read more ...


Modifying Test Signal Generation

One of the tasks I did this week was modify the cti_design_matrix again, as asked by my mentor to make the code more readable. The initial code was following pep8 standard but it wasn’t very easy to read, but now it is. Also, I realized that the main reason my signals weren’t matching the ground truth values before at all was because the eigenvalues and eigenvectors of the diffusion tensor distribution were wrong. This was because, before I tried getting D_flat by doing: np.squeeze(from_3x3_to_6x1(D)) which returned a tensor of shape ( 6, ). But in this case, it returned the diffusion tensor elements in the order : Dxx, Dyy, Dzz and so on which isn’t the correct format of input expected for the “from_lower_triangular” function. So, initially, we were doing : evals, evecs = decompose_tensor(from_lower_triangular(D_flat)) where the from_lower_triangular function is returning a tensor of shape: (3,3). But then I realized that rather than calculating D_flat, we can simply do: evals, evecs = decompose_tensor(D_flat). Following this approach gave the correct value of “evals and evecs”. So, now we have the correct values of “evals and evecs” which made the signals come closer to the ground truth signals, but we still don’t have the signals completely matching the ground truth signals. Another problem we realized was that while passing “C”, covariance tensor parameters, we needed to make sure that we were passing the modified C parameters, that is “ccti”. This again helped in bringing the signals to the expected values. So, after talking things through with my mentor, and analyzing the QTI paper, we came to a few conclusions which could be done to improve the signal values.

Read more ...


Diffusion Model results on pre-trained VQVAE latents of NFBS MRI Dataset: Week 6 & Week 7

My current code for VQVAE & DM is well tested on MNIST dataset as shown in the previous blog posts. I extended the current codebase for MRI dataset by using 3D convolutions instead of 2D ones, which resulted in 600k parameters for VQVAE for a downsampling factor f=3. I used a preprocess function to transform MRI volumes to the desired shape (128,128,128,1) through DIPY’s reslice and scipy’s affine_transform functions, followed by MinMax normalization. I trained the VQVAE architecture for batch_size=10, Adam optimizer’s lr=2e-4, 100 epochs. I followed suit for downsampling factor f=2 as well and got the following training curves-

Read more ...


Design Matrix Implementation and Coding with PEP8: Week 5

This week, my work focused on two main areas: improving the design matrix and implementing methods under the Fit class in CTI. For the design matrix improvement, I noticed that the design matrix I had previously created was not according to PEP8 standards. After some effort, I managed to modify it to comply with the appropriate format. This week, my time was mostly consumed by implementing methods under the Fit class in CTI. As CTI is an extension of DKI and shares similarities with the QTI model, I had to look into methods already implemented in DKI and QTI. My approach involved going through these two different modules, comparing the methods, and making notes on which ones would need to be implemented in CTI. This was challenging, as CTI’s design matrix is significantly different. Although this implementation is not completely done, I was able to learn a lot.

Read more ...


Creating signal_predict Method: Testing Signal Generation

This week, I worked together with my mentor to come up with a new way of arranging the elements of the design matrix. So, first I rearranged all the parameters of the covariance parameters so that they’d match with the ones in QTI. So now, the order is: the diffusion tensor, the covariance tensor, and then the kurtosis tensors. But then we decided that it would be better to put the kurtosis tensors first because then we wouldn’t have to re-implement all the kurtosis methods again. So, I changed the order of kurtosis and the covariance tensors.

Read more ...


Carbonate Account Setup, Experiment, Debug and Repeat: Week 5

I finally got my hands on IU’s HPC - Carbonate & Big Red 200. I quickly set up a virtual remote connection to Carbonate’s Slate on VS Code with Jong’s help. Later, I started looking up on Interactive jobs on Carbonate to have GPUs on the go for coding and testing. I spent a ton of time reading up on Carbonate’s Interactive SLURM jobs information. Using X11 forwarding, I was able to spin up an interactive job inside the login node using command prompt. It popped up a Firefox browser window from the login node ending up slow and not very user friendly. Same goes for the Big Red 200 as well. Eventually my efforts were in vain and I resorted to installing a jupyter notebook server on my home directory. Although I can’t request a GPU with this notebook, it allows me to debug syntax errors, output visualization, plotting loss values etc.

Read more ...


Re-Engineering Simulation Codes with the QTI Model and Design Matrix

I had to change the cti_test.py file as the signals generated were not exactly correct. I was advised to follow the multiple gaussian signal generation method. While doing this I had to look closely at several already implemented methods and go in depth to understand how those functions were achieving the desired output. The multiple gaussian signal generation method is preferred because the CTI signal generation closely resembles the multiple gaussian signals. We’re using the multiple gaussian signals so that we can have a priori of what to expect from the outcome, if we fit our model to this signal. I also managed to implement the design matrix for the CTI tensor and managed to save it up in the utils.py file. The design matrix is a crucial component of the CTI tensor as it represents the relationships between the different variables in our model. By accurately modeling these relationships, we can generate more realistic simulations and gain a deeper understanding of the CTI tensor. The link of my work: Here <https://github.com/dipy/dipy/pull/2816>__

Read more ...


Diffusion research continues: Week 4

As discussed last week, I completed researching on StableDiffusion(SD). Currently we’re looking for unconditional image reconstruction/denoising/generation using SD. I completed putting together keras implementation of unconditional SD. Since I couldn’t find an official implementation of unconditional SD code, I collated the DDPM diffusion model codebase, VQ-VAE codebase separately.

Read more ...


CTI Simulation and QTI tutorial : Week 3

This week I worked on finishing the simulations with the appropriate documentation. I also worked on creating a general tutorial for CTI/ QTI as one doesn’t already exist for QTI. The idea behind this general tutorial was that there isn’t any tutorial for advanced diffusion encoding. The closest documentation QTI has is here. However, there are several youtube videos. So, in this tutorial we started with simulating qti, and then we make things a little more complex by adding information on CTI as QTI can only handle a single Gradient Table whereas CTI can handle multiple Gradient Tables. This week I also started by initializing cti_tests.py file by adding relevant simulations to it.

Read more ...


VQ-VAE results and study on Diffusion models : Week 3

I continued my experiments with VQ-VAE on MNIST data to see the efficacy of the Prior training in the generated outputs. The output of the encoder for every input image delivers a categorical index of a latent vector for every pixel in the output. As discussed in the previous blog post, the Prior has been trained separately using PixelCNN (without any conditioning) in the latent space.

Read more ...


Signal Creation & Paper Research: Week2 Discoveries

I worked through this research paper, and found some relevant facts to the tasks at hand, such as the different sources of kurtosis. One other important fact I found out was that DDE comprises 2 diffusion encoding modules characterized by different q-vectors (q1 and q2 ) and diffusion times. This fact is important because, CTI approach is based on DDE’s cumulant expansion, and the signal is expressed in terms of 5 unique second and fourth-order tensors. I also found out about how the synthetic signals could be created using 2 different scenarios, which comprises a mix of Gaussian components and a mix of Gaussian and/or restricted compartments. The major time I spent this week was in creating synthetic signals, and therefore in creating simulations.

Read more ...


Deep Dive into VQ-VAE : Week 2

This week I took a deep dive into VQ-VAE code. Here’s a little bit about VQ-VAE -

Read more ...


Community bonding and Project kickstart : Week 1

Community Bonding period ended last week and my first blog is based on the work carried out in the last week. My meeting with GSOC mentors at the start of the week helped me chalk out an agenda for the week. As the first step, I familiarized myself with Tensorflow operations, functions and distribution strategies. My previous experience with PyTorch as well as website tutorials on basic Deep Learning models helped me quickly learn Tensorflow. As the next step, I read VQ-VAE paper & understood the tensorflow open source implementation. VQ-VAE addresses ‘posterior collapse’ seen in traditional VAEs and overcomes it by discretizing latent space. This in turn also improved the generative capability by producing less blurrier images than before. Familiarizing about VQ-VAE early on helps in understanding the latents used in Diffusion models in later steps. I also explored a potential dataset - IXI (T1 images) - and performed some exploratory data analysis, such as age & sex distribution. The images contain entire skull information, it may require brain extraction & registration. It maybe more useful to use existing preprocessed datasets & align them to a template. For next week, I’ll be conducting further literature survey on Diffusion models.

Read more ...


Community Bonding and Week 1 Insights

Hey there! I’m Shilpi, a Computer Science and Engineering undergrad at Dayananda Sagar College of Engineering, Bangalore. I’m on track to grab my degree in 2024. My relationship with Python started just before I started college - got my hands dirty with this awesome Python Specialization course on Coursera. When it comes to what makes me tick, it’s all things tech. I mean, new technology always excites me. Ubuntu, with its fancy terminal and all, used to intimidate me at first, but now, I get a thrill out of using it to do even the simplest things. Up until 2nd year I used to do competitive programming and a bit of ML. But from 3rd year I’ve been into ML very seriously, doing several courses on ML as well solving ML problems on Kaggle. ML is very fun and I’ve done a few project on ML as well. Coding? Absolutely love it. It’s like, this is what I was meant to do, y’know? I got introduced to git and GitHub in my first year - was super curious about how the whole version control thing worked. And then, I stumbled upon the world of open source in my second year and made my first contribution to Tardis: (tardis-sn/tardis#1825) Initially, I intended on doing GSoC during my second year but ended up stepping back for reasons. This time, though, I was fired up to send in a proposal to at least one organization in GSoC. And, well, here we are!

Read more ...


Journey of GSOC application & acceptance : Week 0

While applying for the GSOC 2023 DIPY sub-project titled “Creating Synthetic MRI”, I knew this would be the right one for me for two reasons. Keep reading to know more!

Read more ...