Angel

@angelcrypto

83 Following

19 Followers

Angel pfp

Been a while since I've been on here. A lot going on, but now I'm ready to start a new series, and I will finish it this time! The new series will be about Cuda C/C++ programming! I've always wanted to learn this, so why not share my learning journey in here. Will start soon!

0 reply

0 recast

0 reaction

Angel pfp

### Automatic Differentiation in C++ ### Day 9: Adding more arithmetic operations I've been busy last week but I'm back now. I've added some basic arithmetic operations to allow interactions between tensors and primitive C++ numbers (float, double, int...)., e.g. doing somthing like `tensor + 3`

2 replies

0 recast

0 reaction

Angel pfp

### Automatic differentiation in C++ ### Day 8: Forward mode implementation part 2 Done with the inital implementation of FM. It's a basic one with no real optimizations. I'm working on making it faster. You can find it on github. I'll share an explanation later today!

4 replies

0 recast

1 reaction

Angel pfp

Quick update, I've decided to make the implementation consider the input vector as multiple inputs (x_1, x_2..., x_n), seems to be what's FM is intended to be implemented as. I'll finish it today hoepfully and share the push the code to git soon!

2 replies

0 recast

0 reaction

Angel pfp

When treating the input as a single input vector, forward mode would compute dy/dx in one pass, and if you treat it as multiple inputs, the forward mode will compute it dy/dx_i for each value in that input, thus requiring multiple forward passing. Implementation wise, I'll have to think which way is better.

2 replies

0 recast

0 reaction

Angel pfp

On a side note, I'm still trying to get used to vim motions, my mouse reflections still kicks in whenever I want to change files or scroll through the code 😂 I'm liking the hjkl movements as well as the basic insertion and cursor movements shortcuts, vey nice!

1 reply

0 recast

0 reaction

Angel pfp

I had some cool small examples of the manual execution and values of the gradients trace, but it got deleted somehow 🥲 I'll do it again and share it the next day!

2 replies

0 recast

0 reaction

Angel pfp

### Automatic differentiation in C++ ### Day 8: Forward mode implementation part 1 I've figured out how to implement the forward mode for the simple cases: Case 1: input x is a vector and output y is a real number. The second case is just a generalization of the output (y is also a vector).

7 replies

0 recast

0 reaction

Angel pfp

Thank youuuu! Please let me know if you have any feedback 🙏

1 reply

0 recast

0 reaction

Angel pfp

I've pushed my code on the github rep btw for anyone interested! Next up is to implement the forward mode with dual numbers!

1 reply

0 recast

0 reaction

Angel pfp

Also, I've set up vim motions on vscode 😢 Let's see how much time will it take me to get used to it and see if it really does help with productivity (may the vim gods be with me 🙏 .

3 replies

0 recast

0 reaction

Angel pfp

### Automatic DIfferentiation in C++ ### Day 7: Testing basic operations, refreshing some math and learning vim motions Tested my basic tensor operations (works 😁 ). Also, tried to apply simple chain rule on some functions just to make sure I remember how to do it😅

5 replies

0 recast

0 reaction

Angel pfp

### Automatic Differentiation in C++ ### Here's the github repo (empty now), I'll add my current code there tonight! https://github.com/angel-cr/AutoGrad

0 reply

0 recast

0 reaction

Angel pfp

Yes I did, thank you so much!!

0 reply

0 recast

0 reaction

Angel pfp

Just had a day off from doing coding today, so no new things done for the project. I'll be back tomorrow!

5 replies

0 recast

0 reaction

Angel pfp

### Automatic Differentiation in C++ ### DAY 5: Simple Tensor class implementation: Part 1 Started the Tensor implementation. As mentioned before, this will be 1d vector base implementation of a Tensor for now for training purposes. Finished the basic operations and some math ones (exp, tang, sin, cos).

3 replies

0 recast

0 reaction

Angel pfp

I'll share the github repo later today or tomorrow morning at most!

1 reply

0 recast

0 reaction

Angel pfp

Consider this as C++ training until I get a bit more used to it, and then I'll build on the 1d project and generalize it.

1 reply

0 recast

0 reaction

Angel pfp

### Automatic differentiation in C++ ### Day 4: Today I started working on my Tensor class (rip 🤕) .I thought a little bit about what do I want it to have at first and I figured that since I'm still (re)starting with C++, I'll make it simple: it will support 1d vectors for now and some basic operations.

1 reply

0 recast

0 reaction

Angel pfp

Also, I figuered I'll write my own little Tensor class as well, something similar to the pytorch one (but with less functionalities of course). I'll go on afterwards to add both modes of AD, starting with forward mode. Tomorrow, I'll start implementing that and start sharing real progress (and code!).

1 reply

0 recast

0 reaction