Author: Ayan Pahwa

Categories
Community

A Deep Dive into DeepSeek and the Generative AI Revolution

deep dive into deepseek

If you’ve been anywhere near the tech world in the past year, you’ve probably noticed that Generative AI is the talk of the town. From writing code to generating art, AI models are reshaping how we think about creativity, productivity, and problem-solving. But with so many models out there, it’s easy to get lost in the noise. 

As a developer community leader at Developer Nation, I often get asked: Where are we now in the AI journey? With the recent launch of DeepSeek, it’s time to take stock, explore the landscape, and see how this new contender reshapes the field.Today, we’re going to break it all down, explore the latest entrant in the AI race—DeepSeek—and see how it stacks up against the heavyweights like OpenAI’s GPT and Meta’s Llama.

So, grab your favorite beverage, sit back, and let’s dive into the fascinating world of AI models!

The AI Landscape: Where Are We Now?

In 2025, AI isn’t just a buzzword; it’s an integral part of our lives. The AI landscape is like a bustling metropolis, with new skyscrapers (read: models) popping up every few months. At the heart of this city are Generative AI models, which have evolved from simple text predictors to sophisticated systems capable of understanding context, generating human-like text, and even coding.

Here’s a quick snapshot of where we stand:

  1. OpenAI’s GPT Series: The undisputed king of the hill. GPT-4 is the latest iteration, known for its versatility, massive context window, and ability to handle complex tasks like coding, content creation, and even passing exams.
  1. Meta’s Llama: The open-source challenger. Llama (Large Language Model Meta AI) is designed to be more accessible and efficient, making it a favorite among developers who want to tinker with AI without breaking the bank.
  1. Google’s Bard: Google’s answer to GPT, Bard is integrated with Google’s vast ecosystem, making it a strong contender for tasks that require real-time data and web integration.
  1. Anthropic’s Claude: Focused on safety and alignment, Claude is designed to be more “helpful, honest, and harmless,” making it a popular choice for applications where ethical considerations are paramount.

And now, entering the stage is DeepSeek, a new player that promises to shake things up. But before we get into DeepSeek, let’s take a quick detour to understand what goes into making a Generative AI model.

Before proceeding, take 10 seconds to subscribe to our newsletter where we share a plethora of new resources to your mailbox twice every week so you can stay ahead in the game. 

The Anatomy of a Generative AI Model

Building a Generative AI model is like assembling a high-performance race car. You need the right engine, fuel, and tuning to make it go fast and handle well. Here’s a breakdown of the key components:

  1. The Engine: Neural Networks
    At the core of every Generative AI model is a neural network, typically a Transformer architecture. These networks are designed to process sequential data (like text) and learn patterns by adjusting weights during training.
  1. The Fuel: Data
    The quality and quantity of data are crucial. Models are trained on massive datasets—often terabytes of text from books, websites, and other sources. The more diverse and high-quality the data, the better the model’s performance.
  1. The Tuning: Training and Fine-Tuning
    Training a model involves feeding it data and adjusting its parameters to minimize errors. Fine-tuning is where the magic happens—specialized datasets are used to adapt the model for specific tasks, like coding or customer support.
  1. The Nitrous Boost: Compute Power
    Training these models requires insane amounts of compute power. Think thousands of GPUs running for weeks or even months. This is why only a few organizations have the resources to build state-of-the-art models.
  1. The Steering Wheel: Prompt Engineering
    Once the model is trained, how you interact with it matters. Prompt engineering is the art of crafting inputs to get the desired output. It’s like giving the AI clear directions to navigate the vast landscape of possibilities.

It’s not all sunshine and roses. The current landscape has three major pain points:

  1. Data Requirements: Generative AI models are hungry for data—a colossal amount of it.
  2. Compute Costs: Training and fine-tuning state-of-the-art models can burn through millions of dollars in compute.
  3. Generalization vs. Specialization: Many models are generalists. While they can write poetry and code, they often fall short in domain-specific tasks.

Enter DeepSeek—a new generative AI model that claims to address these issues while bringing unique capabilities to the table. But before we dive into DeepSeek, let’s pull back the curtain on how generative AI models like these are built. Now that we’ve got the basics down, let’s turn our attention to the star of the show—DeepSeek.

DeepSeek: The New Kid on the Block

DeepSeek is the latest entrant in the Generative AI space, and it’s making waves for all the right reasons. But what exactly is DeepSeek, and how does it differentiate itself from the competition?

What is DeepSeek?

DeepSeek is a state-of-the-art Generative AI model designed to excel in code generation, natural language understanding, and creative tasks. It’s built with a focus on efficiency, scalability, and developer-friendly APIs, making it a compelling choice for software developers.

What Can DeepSeek Do?

  1. Code Generation: DeepSeek can generate high-quality code snippets in multiple programming languages, making it a powerful tool for developers looking to speed up their workflow.
  1. Natural Language Understanding: Whether it’s answering questions, summarizing text, or generating content, DeepSeek’s language capabilities are on par with the best in the industry.
  1. Creative Tasks: From writing poetry to generating marketing copy, DeepSeek’s creative abilities are impressive, thanks to its fine-tuning on diverse datasets.
  1. Customizability: DeepSeek offers robust APIs and tools for fine-tuning, allowing developers to adapt the model to their specific needs.

What Makes DeepSeek Different?

  1. Efficiency: DeepSeek is designed to be more resource-efficient, meaning it can deliver high performance without requiring massive compute resources.
  1. Developer-Centric: DeepSeek’s APIs and documentation are tailored for developers, making it easier to integrate into existing workflows.
  1. Scalability: Whether you’re a solo developer or part of a large team, DeepSeek’s architecture is built to scale with your needs.
  1. Openness: While not fully open-source, DeepSeek offers more transparency and flexibility compared to some of its competitors, giving developers more control over how they use the model.

DeepSeek vs. GPT vs. Llama: The Showdown

Now, let’s get to the fun part—how does DeepSeek stack up against the titans of the AI world, OpenAI’s GPT and Meta’s Llama?

FeatureDeepSeekGPT-4Llama
Code GenerationExcellentExcellentGood
Natural LanguageStrongBest-in-classStrong
EfficiencyHighly efficientResource-intensiveEfficient
CustomizabilityHighModerateHigh
OpennessMore open than GPTClosedFully open-source
Developer ToolsRobust APIs, easy to useRobust APIs, but complexLimited, but improving

DeepSeek vs. GPT vs. LLAMA: The Showdown

FeatureDeepSeekOpenAI GPTLLAMA
Training EfficiencyClustered Fine-Tuning (40% cost reduction)Expensive, requiring massive computeModerate but not optimized for cost
Domain ExpertiseFocused (e.g., technical, academic)GeneralistGeneralist
API LatencyLow (<100ms)Medium (~200ms)High (~300ms)
ExplainabilityBuilt-in toolsMinimalNone
Community EcosystemNewEstablishedEmerging

What Does This Mean for Developers?

Key Takeaways:

  • DeepSeek shines in efficiency and developer-friendliness, making it a great choice for developers who want a powerful yet accessible AI tool.
  • GPT-4 remains the gold standard for natural language tasks, but its resource requirements and closed nature can be a barrier for some developers.
  • Llama is the go-to for open-source enthusiasts, but it may require more effort to fine-tune and deploy compared to DeepSeek.

Wrapping Up: The Future of AI is in Your Hands

The AI landscape is evolving at breakneck speed, and DeepSeek is a testament to how far we’ve come. Whether you’re a seasoned developer or just starting out, tools like DeepSeek, GPT, and Llama are opening up new possibilities for innovation and creativity.

So, what’s next? The future of AI is not just about bigger models—it’s about smarter, more efficient, and more accessible tools that empower developers like you to build the next big thing. And with DeepSeek entering the fray, the race is only getting more exciting.

What do you think about DeepSeek? Will it dethrone GPT, or is Llama still your go-to? Let us know in the comments below, and don’t forget to share this post with your fellow developers. Until next time, happy coding! 🚀

P.S. If you’re itching to try out DeepSeek, head over to their website and get started with their developer-friendly APIs, and if you wanna stay closely connected to tech eco-system then don’t forget to subscribe to our Newsletter, Trust us, your inner coder will thank you 😉

Categories
Community

What’s Cooking in the 29th edition of Developer Nation survey: A Letter to Our Community

Your Insights, Big Prizes, and a Greater Impact this time!

Hey Developers,

Today, I’m not just writing out to you as a member of the Developer Nation community but as someone who genuinely believes in your power—yes, your power—to shape the future. 29th Developer Nation isn’t just another survey; it’s a rallying cry for a collective purpose, a testament to what we can achieve together. So let’s talk about you, your role, and how together we can make something extraordinary.

Your Voice, Our Future

Every line of code you write, every system you build, every idea you bring to life—it’s not just technology. It’s a story, your story, woven into the fabric of something bigger. The 29th Developer Nation survey is about capturing those stories. This isn’t about data points or graphs; it’s about You. The technologies you’re mastering, the challenges you’re overcoming, and the dreams you’re building.

When you participate, you’re not just filling out a survey—you’re sharing your journey. We’re exploring the most dynamic tech domains:

–  Backend/Cloud: The invisible architects of modern apps.

–  Internet of Things(IoT): Connecting dreams to reality, one device at a time.

–  3rd-Party Ecosystems: Enabling limitless possibilities.

–  AR/VR: Turning imagination into immersive experiences.

–  Embedded Systems: The unsung heroes where hardware meets software.

We need your voice because your insights are the foundation of what comes next.

The Heart of Our Community

In DN27, over 34,000 developers stepped up, and together, we achieved over 10,000 usable responses. It wasn’t just a number; it was a movement. A community-wide embrace of what we can accomplish when we come together. Now, as we aim to surpass 20,000 participants, I want you to imagine the ripple effect—what your contribution can mean to thousands of developers worldwide.

Your Impact, Amplified

Every response to Developer Nation is more than an insight; it’s a contribution to causes that resonate deeply with our shared community:

– Power Coders: Offering refugees a lifeline through tech education.

– Women in Technology: Breaking barriers and building bridges.

– Cancer Research: Accelerating breakthroughs to save lives.

You’re not just answering questions; you’re changing lives. You’re showing the world what it means to be a developer with purpose.

A Token of Gratitude

Let’s be real—we love to give back. As a small thank you, you could win prizes that celebrate your craft and creativity:

– 🎙️ RODE NT-USB Mini Microphone

– 🛸 Imperial Star Destroyer™ Lego Set

– 🍓 Raspberry Pi 5s

– 💻 GitHub Team & Docker Pro subscriptions

– 🔒 Mullvad VPN licenses

– 🎁 Gift Cards & more

These aren’t just giveaways; they’re a celebration of you and your passion.

A Call to Action

Imagine this: a world where every developer’s voice is heard, where every insight sparks innovation, and where every story inspires change. That’s the world we’re building together. But we can’t do it without you. We need you—your perspective, your experience, your passion.

START HERE

Let’s make this the most impactful survey yet together. Because when developers unite, there’s nothing we can’t build. If you haven’t already done so, take a moment to join the community.

Categories
Community

Developer Nation visited API World & Dev Innovation Summit, Santa Clara – a Recap.

Earlier this year at Developer Nation, we announced our new mission – Empowering developers and software creators worldwide to shape the future of technology through research and community. We also decided to take things away from the screens and meet our community where they are. Hence, I attended the API World and Dev Innovation Summit 2024 in Santa Clara this November. It was an incredible experience, and I’m excited to share some highlights with you!  

The Venue and Organization  

First, the venue (Santa Clara Convention Centre) was simply impressive! It was right next to the famous Levis stadium of Santa Clara. Everything was super well-organized, from the registration process to the booth layout, making navigating and engaging with the event easy. A thoughtful touch was the QR codes on attendee badges, which made post-event connections seamless. If you’ve ever had trouble remembering who you spoke with after an event, this was a game-changer!  

Booths and Swag 

The booth area was another standout feature—perfectly organized, with companies showcasing creative solutions to help developers tackle challenges across the API development and management lifecycle. And let’s talk about the swag! From soft plushies and custom Lego kits to T-shirts and stickers, the booths truly brought their A-game. Tesla also had a booth, and I got to test drive their Cybertruck. How crazy is that xD. 

Talking about Swags and Prizes, we’re giving away a ton of them. All you have to do is participate in our 29th Developer Nation survey and win prizes like Rode NT USB microphone, Lego kits and more. Participate now, and let’s shape the future of technology together. 

Talks, Demos, and AI Conversations 

Some of the talks were incredibly resourceful, providing insights we can apply daily. I especially enjoyed demos by companies like Traefik and Hasura. Artificial Intelligence was immersed everywhere this year; almost all the booths I stopped by had something to share about AI, a feature they recently added to their product or something they’re considering in the future. It was also fascinating to hear contrasting opinions on AI —some optimistic, others cautious—but all thought-provoking. Discussions around AI added depth to the event and sparked plenty of conversations.  

Networking and Connections 

One of the best parts of API World was meeting interesting people and forming meaningful connections. The event created a great environment for networking, and I walked away with not just new contacts but new perspectives as well.  

Dedicated Media Boxes  

The dedicated media boxes for recording interviews and podcasts were a huge plus for those who create content. They made it easy to capture interviews and sessions without the typical background noise of an event.  Unfortunately, being a solo rider this time, I couldn’t make the most out of it, but I’m surely looking forward to next time when another team member could join me during the conference. 

What I’d Love to See Next Time  

While the event was fantastic overall, there’s always room for improvement. Here’s what I’d love to see next year:  

– More technical talks with a balanced mix of hands-on sessions and sponsor tool walkthroughs.  

– More panel discussions and fireside chats on emerging topics to encourage diverse perspectives.  

– Opportunities for developers to present topics on the fly or share their thought leadership.  

Final Thoughts 

Attending API World 2024 was a rewarding experience, both professionally and personally. It’s inspiring to see so many companies working to make developers’ lives easier, and the event reinforced the importance of our shared mission at Developer Nation.  

If you’re curious about any of the tools or ideas discussed at the event, feel free to reach out—I’d love to chat about them! Let’s keep building and shaping the future of technology together.  

Have you attended any events like this recently? Share your experiences in the comments—I’d love to hear about them! 🚀  

-Ayan

Categories
Community

The Ultimate Developer Nation Holiday Gift Guide – 2024  

As the holiday season approaches, it’s time to spread cheer and appreciation with thoughtful gifts again. Developers, engineers, and tech enthusiasts dedicate countless hours to building and innovating, and they deserve a little holiday joy! 

This year’s Developer Nation Gift Guide is packed with crowd-sourced ideas from our team & community, carefully curated to make any developer’s holiday season brighter. Whether you’re shopping for a colleague, a loved one, or treating yourself, this list has something for everyone.  

{{ advertisement }}

1. Keychron Mechanical Keyboard

Nothing beats the tactile joy of a mechanical keyboard, and Keychron delivers an unbeatable blend of aesthetics and performance. A perfect gift for developers who spend long hours coding—they’ll love these ergonomic keyboards’ satisfying *click-clack*.  Our community members love it, and we’re working with Keychron for a surprise treat, which we will announce soon. Subscribe to our newsletter and stay tuned for more 😉

—

2. Logitech MX Master 3 Mouse

Another favourite in our community is this ultimate productivity mouse, beloved by developers for its ergonomic design and precision. Its seamless multi-device connectivity and programmable buttons make it a game-changer for multitasking engineers. 

—

3. YubiKeys 2FA USB Hardware Security Keys

Give the gift of digital security. YubiKeys offer top-notch two-factor authentication, ensuring developers can easily safeguard their online accounts.  

—

4. Sticky Weekly Calendar Notes

For the organized developer—or one who could use a little help staying on top of tasks—these sticky notes bring analogue charm to their otherwise digital workspace.  

—

5. Loop Switch Earplugs – Multi-Mode Noise-Reducing Earplugs

Help developers block out distractions or enjoy a peaceful environment. These stylish earplugs offer adjustable noise reduction, which is ideal for deep work or noisy commutes.  

—

6. Udemy Course Vouchers

Fuel their passion for learning! With Udemy vouchers, developers can explore new technologies, enhance skills, or explore personal interests.  

—

7. Collapsible Water Bottle

Stay hydrated and save desk space with this eco-friendly, portable water bottle. Perfect for devs on the go or those glued to their screens.  

—

9. Rode NT USB Mini Microphone 

For developers dabbling in podcasts, streaming, or just crystal-clear Zoom calls, this compact microphone ensures they sound professional. Also the mic I use to record Developer Nation Broadcast Podcast 😉

Rode NT-USB Mini Microphone

—

10. GitHub Team and Docker Pro Subscriptions 

Boost their workflow with a subscription to GitHub Team or Docker Pro. Ideal for developers looking to optimize collaboration and containerization in their projects. We’re giving them away to our survey winners; participate here to Enter.   

—

11. Mullvad VPN License 

Privacy matters, especially to developers. Mullvad VPN is an excellent choice for those who value online security and anonymity.  

—

12. Circuit Playground Express Board 

Perfect for hardware enthusiasts, this board makes experimenting with circuits and coding easy and fun. Great for building creative projects. I highly recommend this for anyone who wants to get started learning Embedded Systems using MicroPython or CircuitPython.

—

13. *The 4-Hour Chef* book by Tim Ferriss

More than a cookbook, this book teaches rapid learning techniques through the lens of cooking. A unique gift for curious minds who love to hack their way to mastery.  

—

14. Desk Plants 

Bring a touch of green to their desks! Plants brighten up a workspace and boost mood and productivity—just make sure it’s a low-maintenance variety.  

—

15. 1TB NVME M.2 SSD 

Because you can never have too much storage. A lightning-fast SSD is a practical gift that any developer will appreciate. Don’t forget to pick a case with USB C to use as a portable high-speed SSD/flash drive.

—

16. Programmer Desk Coasters 

Celebrate developer humour with these quirky coasters featuring cheat sheets for tools like Vim. A fun, functional addition to their workspace. Check them out here  

—

17. Nintendo Switch

For gaming after hours or while on the road to work! The Switch is a perfect gift for developers who enjoy unwinding with a bit of fun and nostalgia.  

—

18. Rio Grande Games Dominion Second Edition

Unleash their strategic side with this award-winning deck-building card game. A fantastic gift for game-loving devs who enjoy offline challenges.  

—

19. Victorinox SD Multitool

Small, sleek, and endlessly useful. This multitool is perfect for quick fixes, both in the office and on the go. My personal favourite when it comes to fixing everyday errands.

—

20. G-Shock GSA 2100

The rugged, stylish G-Shock watch is perfect for developers who want durability and modern aesthetics in one package.  

—

Wrapping Up 

This holiday season, show your appreciation with gifts that blend practicality, fun, and geeky charm. Whether it’s improving their workflow, sparking creativity, or encouraging downtime, there’s something here to make every developer smile.  

Wishing you all a joyous holiday season filled with warmth, gratitude, and maybe a few lines of celebratory code. Happy gifting!  

—

What’s on your gift list this year? Share your ideas in the comments below! 🎁

Categories
Community

Accelerating IoT development using Cloud workflow with Arm virtual hardware

Having spent significant time in the realm of embedded systems and IoT development, I’ve consistently encountered challenges related to initial setup and scaling to production. Chip selection, a pivotal aspect of this process, often involves meticulous shortlisting of microcontrollers and architectures, followed by the acquisition of development boards and prototyping to identify the most suitable chipset. Not to mention the lead times and chip shortage problems, this is merely the beginning.

Once this initial step is complete, the journey extends to establishing IDEs, debuggers, test environments, and other development, testing, collaborating and shipping tools. This process can be arduous, difficult to scale, and often discouraging. Yet, as those familiar with my work know, I’ve consistently been vocal about the bridging of the gap between hardware/IoT/embedded and software/cloud development, thanks to new product lifecycle management workflows and DevOps practises.

Cloud developers have long enjoyed the luxury of robust tools and streamlined software lifecycles. The ability to scale from a single server instance to thousands with a simple click, facilitated by Docker Containers, Kubernetes, and DevOps workflows like CI/CD, has been a major draw for embedded developers.

One such innovation that brings IoT developers closer to this cloud-native model is Arm’s Virtual Hardware on the cloud. Let’s talk more about that in this blog and see how it fits into the bigger picture starting from:

Pain Points in Traditional Embedded and IoT Development

Developing for embedded and IoT applications involves numerous challenges, including:

  • Hardware Dependencies: Reliance on physical hardware can significantly delay development cycles, as changes often require new hardware or software modifications.
  • Long Shipping Delays and Chip Shortages: Procuring specialised hardware can be time-consuming, especially during chip shortages (a major impact seen during the COVID-19 pandemic), hindering development progress.
  • Limited Testing Environments: Testing embedded software on physical hardware can be resource-intensive and requires specialised custom jigs, debuggers, physical access, Test and Measurement equipment and much more.
  • Integration Challenges: Coordinating hardware and software development teams can be difficult, leading to delays and potential integration issues.

How Arm Virtual Hardware Changes the Game

Arm Virtual Hardware addresses these challenges by providing a virtualised environment where developers can simulate and test embedded and IoT applications without relying on physical hardware. This offers several key benefits:

  • Faster Time to Market: AVH accelerates development cycles by enabling testing and debugging early in the process, reducing the time it takes to bring products to market.
  • Development Without Hardware: Developers can start working on their applications before physical hardware is available, saving time and resources, which is very helpful when you’re in the process of deciding the hardware or ordering dev boards.
  • Bridging the Gap Between Hardware and Software: AVH fosters collaboration between hardware and software teams by providing a common platform for testing and integration.
  • Enabling Cloud Workflows and DevOps: Thanks to AVH, you can now use most of the tools cloud developers have to ship products faster into the market. Virtual hardware can be integrated into cloud-based development environments, enabling DevOps practices and continuous integration/continuous delivery (CI/CD) pipelines. 
  • Scalable: Firing up 200 new Virtual hardware in the cloud takes 10 seconds; compare this to testing your code in 200 new development boards; the scalability factor with Arm virtual hardware is phenomenal and a breeze regarding regression testing. 

Enabling MLOps with Arm Virtual Hardware

Running ML models on the Edge compute devices is one of the most common applications where Arm-based processors are deployed today. Think of smart speakers, phones, traffic lights, cameras, etc. These products and applications can benefit greatly by adopting Arm virtual hardware in prototyping and testing life cycles. 

Machine Learning Operations (MLOps) involves managing the entire lifecycle of machine learning models, from development to deployment. Using Arm virtual hardware, developers and data scientists can test their model on almost a real Arm processor, estimating the performance of different architectures and chipsets; this enables them to pick the best hardware suitable for their models, saving a lot of cost and time in development and bringing product to market. Developers can train machine learning models on virtualised Arm hardware, ensuring compatibility with target devices and architecture, enabling

  • Testing and Optimization: AVH can test and optimise models for performance and resource utilisation on embedded platforms.
  • Deployment: Once ready, models can be deployed to physical devices with little to no adaptation using AVH as a reference environment.

In Conclusion

Arm Virtual Hardware is a game-changer for embedded and IoT development. By addressing the challenges of traditional development methods, AVH enables faster time to market, improved collaboration, and enhanced flexibility. As the adoption of embedded and IoT devices grows, AVH will play a vital role in driving innovation and efficiency.

Categories
Tips

Edge Computing and Machine Learning Key Insights

Edge Computing Machine Learning

This blog post is powered by data from Developer Nation’s 26th global survey wave (conducted from November 2023 to January 2024), delves into the latest and most crucial developer trends for Q1 2024. With insights from over 13,000 developers across 136 countries, it’s a treasure trove of knowledge.

37% of developers who target non-x86 architectures write optimised code for Arm-based processors, making them the second most popular target behind microcontrollers (40%).

Nearly half (47%) of ML developers have deployed on-device AI solutions in the past 12 months. The top motivations for doing so are increased user privacy and faster inferencing.

The most popular on-device ML framework is Google MLKit, used by 46% of developers who deploy on-device AI solutions, followed by OpenCV (28%), PyTorch Mobile (26%), and TensorFlow Lite (25%).

The vast majority (86%) of developers working on Industrial IoT projects implement on or near-device solutions, with the most popular being on-device processing (26%), automation control (19%), and real-time analytics (17%).

Categories
Community

Advanced git: Demystifying git Remotes and git cherry-pick: Powerful Tools for Collaboration

Collaboration is key in the world of Git version control. But keeping track of changes from multiple developers can get tricky. This blog post dives into two essential Git features—remotes and cherry-pick—that empower you to streamline your workflow and effectively manage contributions.

Understanding Git Remotes: A Bird’s Eye View

By default, your GitHub repository typically has a single remote—the origin, representing the main repository you cloned from. However, in larger projects with multiple developers, things get more interesting. Often, developers create personal forks before they push their code.This allows them to work on a separate copy of the code base, and once they are satisfied with the changes, they can merge back into the main codebase.

Here’s where remotes come into play. They are references to additional copies of your Git repository, potentially containing valuable contributions from other developers.

Let’s use an Open-Source project: Lottie

Imagine we’re working with the fantastic Lottie by Airbnb, a library that renders After Effects animations on mobile platforms. We’ve cloned a fork (iayanpahwa/lottie-android) and want to explore changes made by other contributors to lottie (gpeal and felipecsl).

Adding Remotes: Reaching Out to Other Forks

To access these developers’ workspaces, we can add them as remotes using the git remote add command:

git remote add <remote_name> <repository_URL>

For example:

git remote add gpeal https://github.com/gpeal/lottie-android.git
git remote add felipecsl https://github.com/felipecsl/lottie-android.git

Now, using git remote -v, you can see all configured remotes, including their URLs.

Fetching the Goods: Downloading Changes

With remotes in place, we can retrieve changes from other contributors using git fetch.

  • Fetching from a specific remote:
  • Fetching from all configured remotes:
	git fetch --all

This downloads the commits made by these developers without integrating them into your local working directory yet.

git cherry-pick: Borrowing the Best Bits

Git cherry-pick allows you to meticulously select and apply specific commits from other branches (including those fetched from remotes) onto your current branch. This is particularly useful for integrating contributions from multiple developers, testing them individually, or incorporating specific fixes.

A Real-World Cherry-picking Scenario

Imagine you manage an open-source project that receives a wave of pull requests. You might want to test these contributions together before merging them. Here’s how cherry-picking can help:

Create a New Branch:

git checkout -b my-test-branch
  1. Fetch Necessary Code (if not already done): Use git fetch as explained earlier.
  2. Cherry-picking Commits: Once you have access to the desired commits, cherry-pick them one by one using their commit hashes:
git cherry-pick <commit_hash>

For instance, to test a specific commit (648c61f5275998c461347b5045dc900405306b31) by contributor gpeal:

git cherry-pick 648c61f5275998c461375647845dc900405306b31 [ commit made by gpeal ]

This brings gpeal’s changes to your my-best-branch for isolated testing.

Remember: Cherry-picking can rewrite history, so use it cautiously. Always create a dedicated branch for testing before integrating changes into your main codebase.

Wrapping Up:

By mastering remotes and cherry-pick you can effectively collaborate on Git projects, leverage valuable contributions from others, and ensure a smooth and efficient development workflow.

Feel free to reach out with any questions! Happy coding! Do check our blogs on git internals for more learning: 

Categories
Community

Managing Complex Dependencies with Google’s repo tool

In my last blog, I discussed managing dependencies with git submodules. However, when working with large projects that have many dependencies, traditional methods like git submodules can become cumbersome. Google’s repo tool emerges as a powerful solution specifically designed to handle this challenge.

What is repo tool?

repo is an in-house dependency management tool developed by Google. It excels at managing many dependencies, making it ideal for projects like the Android Open Source Project (AOSP) and custom Android ROMs.

Unlike git submodules, which are an integrated git feature, repo functions as a separate executable script. This necessitates installation before diving in.

Installation (Choose your adventure!)

Linux: 

Create a directory for Repo:

mkdir ~/bin

Update your PATH environment variable:

export PATH=~/bin:$PATH

Download and make Repo executable:

curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
Google repo tool

OSX:

Use Homebrew to install Repo:

brew install repo
Google repo tool

For other platforms, refer to official docs: https://gerrit.googlesource.com/git-repo

Manifest Magic: Defining Dependencies

Repo relies on a manifest file stored in a separate Git repository. This XML file is the central hub, outlining where to fetch project dependencies, their storage location, and specific revisions (commits).

The beauty of Repo lies in its ability to manage multiple manifests. Imagine a huge, complex project like the Android Operating system with 100 dependencies. You could create a dedicated “lib.xml” manifest to fetch those specific libraries, eliminating the need to include hundreds of unrelated dependencies from a broader manifest. Similarly, the testing and compliance team can have “qa.xml” and “compliance.xml” to manage extra QA and compliance-related dependencies separately, which might not be needed in production but required during development. Both could also have the same libraries but different versions. Hence repo using manifest.xml makes handling dependencies extremely flexible. 

For this demo, we’ll keep things simple with a single “default.xml” file.

Creating a Manifest

Clone the Example Repository having our manifest:

git clone git@github.com:iayanpahwa/manifest-demo.git

Examine the default.xml file:
This file specifies the main Project (ex, EazyExit) with two dependencies, FastLED and PubSubClient, along with their corresponding URLs, paths, and revision IDs.

<?xml version="1.0" encoding="UTF-8"?>
<manifest>

<remote fetch="https://github.com/iayanpahwa/" name="EazyExit" />
    
    <project name="FastLED.git" path="lib/FastLED" remote="EazyExit" revision="c1ab8fa86f6d6ecbf40ab7f28b36116a3c931916" />
    <project name="pubsubclient.git" path="lib/PubSubClient" remote="EazyExit" revision="dddfffbe0c497073d960f3b9f83c8400dc8cad6d" />

</manifest>

Note: The manifest allows for various configurations, including project branches and alternative remotes (like Bitbucket or GitLab). Refer to the official documentation for a comprehensive list: https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.md

Putting it All Together: Fetching Dependencies

  1. Push the default.xml file to your GitHub repository (if using the provided example).
  2. Create a project directory (e.g., EazyExit).

Navigate to your project directory and initialise Repo

Google repo tool

3. This command establishes the current directory as your project workspace.

Fetch dependencies using the repo sync command:

4. This command retrieves all dependencies specified in the manifest and stores them according to the defined paths.

By leveraging repo, you can effectively manage many dependencies within a single, streamlined workflow.

Repo empowers you to manage complex dependencies with ease, promoting a more flexible and adaptable development process. Checkout our other blogs on: 

Categories
Community

How Git Submodules Can Save You Time (and Headaches): Taming the Dependency Beast

In software development, we rarely build projects entirely from scratch. We leverage open-source libraries and frameworks to accelerate development and avoid reinventing the wheel. But managing these dependencies can quickly become a tangled mess, especially as projects grow and dependencies multiply.

This blog post explores a simple yet powerful Git feature called git-submodule, which streamlines dependency management and keeps your codebase clean and organised.

Git Submodules

The Downside of the Manual Approach

Many developers resort to simply manually cloning and directly pushing dependency code into their main project’s codebase. While this may seem convenient at first, it creates several challenges:

  • Version Control Issues: Updating dependencies becomes a manual process, increasing the risk of compatibility issues and security vulnerabilities.
  • Upstream Changes: New features or bug fixes in the original library require manual integration, which is time-consuming and error-prone.

Introducing Git Submodules

git submodules allow you to integrate external Git repositories (containing your dependencies) directly into your project. This creates a modular approach with several benefits:

  • Independent Updates: You can update submodules individually without affecting your main project code.
  • Version Tracking: Submodules track the specific commit hash of the dependency you’re using, ensuring consistency and reproducibility.
  • Modular Codebase: Your project remains clean and organised, with dependencies clearly separated from your core code.

Putting Git Submodules into Action

Let’s walk through a practical example. Imagine a project named “submodule-demo” that relies on two libraries:

  • FastLED: A library for controlling LEDs
  • PubSubClient: A library for implementing an MQTT client
Git Submodules

Here’s how to leverage git-submodules to manage these dependencies:

  1. Project Structure: You can create a dedicated directory (e.g., lib) within your project to store dependencies.
  2. Adding Submodules: Use the git submodule add command to specify the URL of the external repository and the desired submodule path:
cd your_project/lib
git submodule add https://github.com/iayanpahwa/FastLED.git
git submodule add https://github.com/iayanpahwa/pubsubclient.git
Git Submodules

This fetches the code from the specified repositories and stores them within the lib directory.

3. Initialising and Updating: Anyone cloning your project can easily initialise and update the submodules using the following commands:

git clone <your_project_URL>
cd <your_project_URL>
git submodule init
git submodule update
Git Submodules

Alternatively, you can use the --recursive flag during cloning to automate these steps:

git clone --recursive <your_project_URL>
Git Submodules

4. Version Control: Git submodules record the specific commit hash used from each dependency. This ensures everyone working on the project uses the same library version, promoting consistency and preventing compatibility issues.

Beyond the Basics:

While submodules default to fetching the latest commit from the dependency’s main branch, you can specify a different branch or commit hash. Refer to the official Git documentation (https://git-scm.com/book/en/v2/Git-Tools-Submodules) for details on advanced usage.

Key Takeaways

By embracing git submodules, you can effectively manage dependencies, improve code organization, and streamline project collaboration. This approach promotes a more modular and maintainable codebase, saving you time and headaches in the long run.

Feel free to explore our other blog posts on Git internals for further insights!

Git Internals Part 1- List of basic Concepts That Power your .git Directory

Git Internals Part 2: How does Git store your data?

Git Internals Part 3: Understanding the staging area in Git

NEW Developer Nation survey is live! Participate, shape the trends in software development, and win big. Start here!

Categories
Developer Nation Broadcast

The state of Enterprise IoT Development and MQTT with Kudzai Manditereza

This episode explores the exciting world of Enterprise IoT with Kudzai Manditereza, a Developer Advocate at HiveMQ. Ayan and Kudzai delve into three key areas:

  1. Enterprise IoT Development: They discuss the unique challenges and opportunities associated with developing and implementing IoT solutions for businesses. This involves exploring various technologies, platforms, and considerations specific to enterprise-grade deployments.
  2. Digital Transformation Across Industries: Kudzai sheds light on how digitization is transforming various industries, from manufacturing and healthcare to logistics and retail. He shared how companies are leveraging IoT to improve efficiency, gain real-time insights, and create innovative products and services.
  3. Kudzai’s Role and MQTT: Ayan and Kudzai explore Kudzai’s role in helping traditional companies navigate their digital transformation journeys and adopt IoT solutions. They discussed how Kudzai leverages his expertise to bridge the gap between traditional business models and the possibilities of IoT. The conversation also touched upon the role of MQTT, a popular messaging protocol, in enabling communication and data exchange within IoT ecosystems.

This episode offers valuable insights for businesses considering or embarking on their IoT journeys. By listening to Kudzai’s expertise and Ayan’s engaging discussion, listeners can gain a better understanding of the potential of Enterprise IoT, its impact across industries, and how it can be leveraged for business growth and innovation.