Categories
Analysis

Machine learning developers and their data

Machine learning developers and their data

The data science (DS), machine learning (ML), and artificial intelligence (AI) field is adapting and expanding. From the ubiquity of data science in driving business insights, to AI’s facial recognition and autonomous vehicles, data is fast becoming the currency of this century.  This post will help you to learn more about this data and the profile of the developers who work with it. 

The findings shared in this post are based on our Developer Economics 20th edition survey, which ran from December 2020 to February 2021 and reached 19,000 developers. 

Before you dive into the data, our new global developer survey is live now. We have a set of questions for machine learning and AI developers. Check it out and take part for a chance to have your say about the most important development trends and win prizes.

It takes all types

The different types of ML/AI/DS data and their applications

We ask developers in ML, AI, and DS what types of data they work with. We distinguish between unstructured data — images, video, text, and audio — and structured tabular data. The latter group includes tabular data that they may simulate themselves. 

With 68% of ML/AI/DS developers using unstructured text data, it is the most common type of data these developers work with; however, developers frequently work with multiple types of data. Audio is the most frequently combined data type: 75-76% of those that use audio data also use images, video, or text. 

“Unstructured text is the most popular data type, even more popular than tabular data”

Given the most popular applications of audio data are text-to-speech generation (47%) and speech recognition (46%), the overlaps with video and text data are clear. Image data, like audio, overlaps heavily with video data: 78% of those using video data also use image data. The reverse dependence isn’t as strong: only 52% of those using image data are also video data users. The top two applications of both these data types are the same: image classification and facial recognition. These are two key application fields driving the next generation of intelligent devices: improving augmented reality in games and underpinning self-driving cars, in home robotics, home security surveillance, and medical imaging technology. 

The types of data ML/AI/DS developers work with

69% of  ML/AI/DS developers using tabular data also use unstructured text data

With 59% usage, tabular data is the second most popular type of data. 92% of the tabular data ML/DS/AI developers use is observed, while the other 8% is simulated. The two most common use-cases for this data is workforce planning — 39% of developers who use simulation do this —  and resource allocation, also at 39%.

Structured tabular data is least likely to be combined with other types of data. Although uncommon to combine this type of data with audio or video data, 69% do combine tabular data with unstructured text data. The top application of both tabular data and unstructured text is the analysis and prediction of customer behaviour. This is the sort of analysis often done on the data nuggets we leave behind when searching on retail websites — these are key inputs to algorithms for natural language and recommender systems.

Keeping it strictly professional?

The professional status of ML/AI/DS developers

The professional / hobbyist / student mix in the ML/AI/DS ecosystem

ML/AI/DS developers engage in their fields at different levels. Some are professionals, others students or hobbyists, and some are a combination of the above. The majority (53%) of all ML/DS/AI developers are professionals — although they might not be so exclusively. 

Of all the data types, audio data has the highest proportion of professional ML/DS/AI developers. 64% of ML/AI/DS developers who use this type of data classified themselves as a professional; and the majority (50%) of these professionals are applying audio data to text-to-speech generation. The high proportion of professionals in this field might be a byproduct of co-influencing variables: audio data is the data type most frequently combined with other types, and professionals are more likely to engage with many different types of data. 

Data types popular with students include image, tabular, and text data. Between 18-19% of developers who work with these types of data are students. There are many well-known datasets of these types of data freely available. With this data in hand, students also favour certain research areas. 

Image classification, for example, is popular with developers who are exclusively students: 72% of those students who use image data use it for this application, in contrast to just 68% of exclusive professionals that do. In applying unstructured text data, 38% of exclusive students are working in Natural Language Processing (NLP), while 32% of exclusive professionals are. As these students mature to professionals, they will enter industry with these specialised skills and we expect to see an increase in the practical applications of these fields, e.g. chatbots for NLP. 

“65% of students and 54% of professionals rely on one or two types of data”

Besides differences in application areas, students, hobbyists, and professionals engage with varying types of data. 65% of those who are exclusively students use one or two types of data, while 61% of exclusively hobbyists and only 54% of exclusively professionals use one or two types. Developers who are exclusively professionals are the most likely to be relying on many different types of data: 23% use four or five types of data. In contrast, 19% of exclusively hobbyists and 15% of exclusively students use four to five types. Level of experience, application, and availability of datasets all play a role in which types of data an ML/AI/DS developer uses. The size of these datasets is the topic of the next section.

Is all data ‘big’?

The size of ML/AI/DS developers’ structured and unstructured training data

The hype around big data has left many with the impression that all developers in ML/AI/DS work with extremely large datasets. We asked ML/AI/DS developers how large their structured and unstructured training datasets are. The size of structured tabular data is measured in rows, while the size of unstructured data — video, audio, text — is measured in disc size. Our research shows that very large datasets aren’t perhaps as ubiquitous as one might expect. 

“14% of ML/AI/DS developers use structured training datasets with less than 1,000 rows of data, while the same proportion of developers use data with more than 500,000 rows” 

The most common size band is 1K – 20K rows of data, with 25% of ML/AI/DS developers using structured training datasets of this size. This differs by application type. For example, 22% of those working in simulation typically work with 20K – 50K rows of data; while 21% of those working with optimisation tools work with 50K – 100K rows of data. 

Dataset size also varies by professional status. Only 11% of exclusively professional developers use structured training datasets with up to 20K rows, while 43% of exclusively hobbyists and 54% of exclusively students use these small datasets. This may have to do with access to these datasets — many companies generate large quantities of data as a byproduct or direct result of their business processes, while students and hobbyists have access to smaller, open-source datasets or those collected via their learning institutions. 

A further consideration is, who has access to the infrastructure capable of processing large datasets? For example, those who are exclusively students might not be able to afford the hardware to process large volumes of data. 

Non-tabular data is a useful measure for comparisons within categories: for example, 18% of image datasets are between 50MB-500MB, while only 8% are more than 1TB in size. The measure doesn’t, however, allow for cross-type comparisons, since different types of data take up different amounts of space. For example, 50MB of video data takes up a considerably shorter length of time than 50MB of audio data. 

The size of unstructured training data ML/AI/DS developers work with

The categorisation of the different data sizes was designed to take into account the steps in required processing power. For most ML/AI/DS developers, we expect that a 1-25GB dataset could be handled with powerful, but not specialised, hardware. Depending on the language and modelling method used, 25GB on disc relates to the approximate upper bound in memory size that this type of hardware could support. 

We see that 26% of ML/AI/DS developers using text data and 41% using video data will require specialised hardware to manage their training. The high level of specialized hardware manifests as a barrier-to-entry: data analysis on these large datasets is beyond an achievable scope without the backing of deep pockets supplying cloud-based technology support or infrastructure purchases.

Want to know more? This blog post explores where ML developers run their app or project’s code, and how it differs based on how they are involved in machine learning/AI, what they’re using it for, as well as which algorithms and frameworks they’re using.

Categories
Analysis

The lasting effects of COVID-19 on how developers work and learn

COVID-19 on how developers work and learn

The COVID-19 pandemic has fundamentally changed the way people work and learn across industries, and developers are no exception to that. Although in the grand scheme of things, our data indicates that many developers have been weathering well the repercussions of an unprecedented crisis, there is much more to tell. 

The findings shared in this post are based on our Developer Economics 20th edition survey, which ran from December 2020 to February 2021 and reached 19,000 developers. Previously we reviewed how developers’ needs were changing due to COVID-19. Now we’ve taken a deep dive into our latest survey data to find which developer groups and regional communities were affected the most by the pandemic and in what ways. 

Before you dive into the data, our new global developer survey is live now. We have updated it with questions relevant for developers in 2021 – check it out and take part for a chance to leave your mark on the upcoming trends and win prizes.

How the COVID-19 pandemic affected the way developers work or study

In our survey, we asked developers to assess the impact of the COVID-19 pandemic on the way they work or learn. In perhaps the most salient category, 7% of developers said that they had lost their job in the aftermath of the pandemic and 9% had dropped out of their studies. Notwithstanding the severity of becoming unemployed in times of crisis, the IT sector is still seen as one of the least impacted sectors in terms of hiring during the global pandemic, with an almost unwavering demand for professionals in software and hardware segments.

How the COVID-19 pandemic affected the way developers work or study

Which developers were impacted the most?

Arguably, not every developer has been affected by their company’s decision to enforce remote working to the same extent. For example, does a working parent who juggles the daily demands of commutes and childcare embrace the opportunity of working remotely as a means of having a better work-life balance? On the other hand, if asked, how many pre-pandemic graduates would have, at the time of their study, agreed to go fully remote and potentially miss out on exploring the rich social life that a university offers to young people? Let’s take a closer look at some of our distinct developer groups to understand which factors have been having the greatest impact on ways of working and learning. 

When looking at how the pandemic affected developers of different experience levels, we find that the more experienced developers were also more affected in the way they work. For instance, 40% of developers with less than one year of work experience say they were unaffected in their ways of working, compared to 35% of developers with six or more years of experience. While the gap is not particularly large, junior developers appear to have switched to remote working to a lesser extent. This may be, in part, due to younger people and new hires wanting to go to the office, get to know their colleagues, and connect with their peers.

Developers working for large organisations were the most likely to go fully-remote during the pandemic

Next, we evaluated COVID-19’s impact with respect to company size. We find that developers working for larger companies were clearly more affected in their ways of working by the pandemic. While 42% of developers in small companies between two and 50 employees say they were not affected, the number plummets to less than 30% for developers in companies of over 50 employees. Large enterprises with more than 5,000 employees have been battling the repercussions of the pandemic at the frontlines; 51% of developers here went fully remote compared to just 29% of developers in companies with between two and 50 employees.

Developers in small companies were less affected by the pandemic.

Note that, except for small companies, switching to fully-remote working was the most likely outcome in our survey. There are good reasons for this: large business organisations are naturally more risk-averse and commonly need large contiguous office spaces that have to be fully closed for all of their employees to effectively contain the spread of the virus. On the other hand, many small companies had more remote-friendly organisational structures to begin with. In particular, start-ups have been known to promote a remote-first culture due to the apparent benefits of lower seed capital and broader options to recruit and pool talents together.

Twice as many developers in Western Europe compared to East Asia went fully remote.

Switching to remote working has been more common in Western regions.

Perhaps it comes as no surprise when looking at different regions to find a substantial gap between the East and the West. Our data shows that the Western regions, such as Western Europe and the Americas, have more readily facilitated remote working for their employees. For instance, 41% of Western European developers went fully remote, as opposed to only 20% in East Asia. This could be due to a combination of different factors. For example, pundits argue that many Asian countries score low for having the technological infrastructure deemed necessary to adapt to remote working conditions, such as having poor home-office equipment or internet connectivity that is sensitive to traffic surges. Yes, social factors may have partly played a role, as higher average household sizes and smaller apartments in emerging regions pose roadblocks to their own for employees to balance their work and home life.

The pandemic took a heavier toll on young learners.

Lastly, we looked at COVID-19’s impact on learners. 39% of 18- to 24-year olds stated that the way they study has not been affected during the pandemic. On the contrary, among those students aged 25 and over, 50% or more were not affected. Thus, an interesting trend emerges here, that especially younger learners had to adapt by becoming partly or fully remote. Our data offers one possible clue for this; younger learners are more likely enrolled in a formal degree program than older learners, who are more likely to be self-taught and are to be found burning the midnight oil with online courses and boot camps that have traditionally fostered remote ways of studying.

Categories
Community Interviews

Interview: What is it like working on open-source game development?

open-source game development

I had a chance to speak with Liam Arbuckle, the acting CTO of the game/web development studio/collective (100% open-source) called Signal Kinetics. Liam is based in Australia. 

What is it that you’re working on?

Right now, we’re working on a citizen science game engine (sort of like Project Discovery in Eve Online, but integrating other games as well). We’re aiming to increase science discovery/contribution for everyone through gaming by allowing people/players to: 

1. Contribute to real-world scientific problems/experiments  

2. Help train ml/dl datasets/algorithms (sometimes through their actions in-game) 

3. Engage with users, especially those in the scientific community (we’re working on a service called Arcadia which is basically a fork of Buddypress that will implement features similar to services like Steam & Facebook Games) 

So you are targeting citizen scientists? Is there a particular age range you are targeting?

I believe information should be free, when I was younger, scientific journal access was expensive, also, there is a lack of engagement with the science community in Australia. I want to create something that can’t restrict a person from the science community due to their age, gender, spending ability etc.

What inspired you to create your Game Engine?

I attended Science hackathons, science and gaming, made mars rover, most recently I contributed to the Open Source Rover by NASA’s Jet Propulsion Laboratory.

What else are you working on?

We’re also working on our own game (with potential partnerships with Savy Soda, as an example, in the pipeline) and hoping to make our experience with Arcadia modular:

 1. Users can contribute to scientific research through playing any game in the world by installing a custom add-on designed by the Arcadia developers for the game.

 2. Users can have a “bank” (similar to the Pokemon Home system) that shows their games library, achievements and item list/screenshots in the Arcadia web app.

There’s no real large gaming community to play games online. I want to build a community, where gamers can share screenshots, there’s an overlay to watch people playing games, I want to make mini-games too, there’s no real limit. I used to game on a Samsung phone which had a play status, I could stream to Discord. I want to expand this idea to non-Samsung games, add a community with no limits – basically information freedom, with no blockage or limits.   

3. Users can choose which games to play.

What are your immediate goals?

Get industry connections.

What type of connections are you looking for?

I’ve made contact with Melbourne-based game companies so I’m on track with that. I’m looking for grants, an investment to work on the blockchain element, get connected with a marketing team, get a few 1,000 players to start off with, and then connect with more on social media.

Right now I don’t have the money to finance, we’ve had people come online to help with the open-source. I’d ideally like to get some consistent engagement rather than have contributors that do the occasional work.

If it wasn’t for Covid-19  I would have moved out of Australia, there are huge problems with setting up in Australia, no grants, no infrastructure for tech companies.

I want to start contributing to established games and engines to gain experience, connections, contribute and potentially expand my team’s vision.  

Are there any particular games that you have in mind?

Minecraft, l would love to contribute to that, I Love that you can make mods. I think Minecaft is crying out for more integrations, so I would love to get connections with Mojang. I’ll take any company that has a level of open-source ethos.

Continue working on the game, however, this requires money. A lot. And I’m not rich! I’m primarily focusing on a media kit that will later be used as the basis for a Kickstarter campaign.   

When do you plan to run the Kickstarter campaign?

I won’t have the game finished before the campaign starts, I want to put together a media kit, assets, I’m going to an incubator to learn how to market the game, understand which social media do we target, and which niche users. I have been involved with other Kickstarter projects and know I can’t be too broad with who I target at first. I think in  3-4 months we will be ready to launch the Kickstarter campaign.

I’ve got a team of about ~20-30 people (with most being external/outside collaborators, there are around 10 people that run the show and contribute on a consistent basis). These people have varying levels of experience in game development, design, and web app construction (among other things).

Are you actively looking for more contributors? If so, what level of experience are you looking for?

I’ll take anything, I won’t say no to anyone, I find that the science community say no, if we say no, we’re just defeating the purpose of the project.

We would prioritise people who have c#, and website building experience. Once you get your base established, then start with junior developers. We don’t want to be too closed, but also we don’t want to be too open and not get work completed. 

We are also working on a partnership with the Swedish Power Metal band Veonity to contribute with us on officially licensed songs for our games and the Arcadia platform  – recording is due to start in July which is very exciting!

Did you know that 34% of game developers use C#?

Interview with Liam Arbuckle

When did your interest in development start?

I love Star Wars, at 12 I went into robotics, and in 2016-2017 I worked to build a physical R2D2. In year 10 I started a computer science class at school. Unfortunately, computer science investment in schools is poor, but I had a good teacher that encouraged younger students who were not yet at the age to attend a class to learn in their breaks. I learned Python, and in year 11 I started working on GitHub, learned Ruby on Rails, Gem. 

I ended year 11 and decided I wanted to start developing. There are no astrophysics courses near to me. You can build games and tell stories from computer science.

How do you make decisions when it comes to your next self-improvement step? Do you look at data, attend conferences?

I attended the recent Atlassian conference. Also, there are 20 of us that meet at a bar regularly to talk about problems, I have joined a few teams and am developing professional skills. 

I pitched to investors last year and got 10,000 AUD but it doesn’t last very long in a startup.

I like to see people in the physical world, go to Python global conferences, learning what’s the newest feature with the project that I can use to my advantage.

Has it been a benefit to have online conferences due to Covid-19?

I would never have been able to afford travel to conferences until this year when I’ve started making money, the online conferences are more accessible.

Before, if you are not fully embedded in a developer community, there is not much incentive to go to in-person conferences, there is a huge cost to fly overseas for a conference, and no guarantee that project of interest will be discussed, no guarantee people that people will help you there. There are more frequent conferences now, by more teams, not just big companies doing them.

Do you have a mentor? Or are you mentoring someone else?

I’m a mentor at the University Codjo, mentoring 14-15-year-olds with Autism / ADSD. For me, the computer sciences teacher was a mentor at school, but I don’t have anyone mentoring me right now. I wouldn’t need a mentor right now for teaching me, rather someone who can structure how I do things, I’m not the best, I’m not perfect, people with experience have given great advice to me.

Do you have any words of wisdom for others thinking of building their own games or game engines?

1. I echo the words of “information wants to be free” if everyone open sources and has no barriers, that would be my ideal world!

2. If you want to make any media, games are great, they engage people, I lose interest in reading novels,  in games, there is so much you can involve other people with, everyone can make their own stories. There’s engagement.

What’s in your toolbox?

  • Unity for most of my games stuff
  • Starship, customisable prompt for my terminal – makes everything look so much cooler. I love customising my devices.
  • GitHub
  • Keybase for communications, encryption and there are git integrations.
  • Notion 
  • Visual studio code 
  • Jira by Atlassian – more of an industry-standard than what I was using before.
  • MacBook M1 for on-the-go stuff, I duel boot with Linux when testing.

How do you work as a distributed team? What tools do you use?

Keybase is the main tool, git commits can be seen in there and there are cool bots and tools you can use. It was also acquired by Zoom which shows that things will be great for global teams.

We also use Facebook messenger or WhatsApp for casual talk.

Git commits can be sent there, cool bots, and tools you can use. Was acquired by zoom, shows that things will be great for global teams.

What do you need right now?

Right now direct partnership with companies is needed, funding is so important. Everyone in the team is paying out of their own pockets. The best way we can succeed is with funding so the Kickstarter will work, with partnerships, it will give our Kickstarter legitimacy. 

If you’re interested in joining forces with Liam and his team either as a developer committed to open-source, or a partner, you can reach Liam via his GitHub profile.

We love to hear your development stories, get in touch to share yours.

Categories
Tips

7 Tips on How to Win In Our Referral Program

7 Tips to win our Referral Program

Our referral program counts over 2,800 developers who are quick, smart, and masterfully spreading the survey with their local developer groups and communities. We know that all of you are very keen on winning the top referral prizes like $ 200, $700 or $1,000 USD. But only the top 50 will score rewards. So we recently spoke with one of our top 3 referrers from the previous survey, and he was cool enough to share a few tips on how to climb the leaderboard faster. Read, learn, and apply!

1. Spread your influence. Twitter is cool but in some countries Whatsapp, Telegram, and Discord developer groups are more eager to back you up and take the survey to support you. Test a mix of different apps and sites.

2. Be intentional. It’s ok if you share the survey occasionally to boost your ranking. But if you want to get to the top, it’s worth planning ahead. Try to be active on specific days or times when you know other developers will be scrolling through their phones. 

3. Learn about the people before posting. If you’re posting in a new developer group, forum, or Slack channel, it’s worth getting to know the vibe of the group first. Some are more casual, some are all about learning, some love memes. When you share the survey with them, try to blend in as a native. 

4. Some people love to hear about the survey prizes, some don’t. That’s ok. You can focus on saying that the survey is actually a good learning opportunity to discover new tools and platforms.

5. Use the promo material we have prepared for you. Feeling a bit confused about how to promote the survey? Don’t know what to say? Pick a few tweets, banners or promo texts & grab your custom promo link to share with other developers.

6. Remind people that they need to complete the survey, so as to count towards your referrals. Perhaps some have started the survey and paused. Give them a gentle reminder to go back and finish answering the questions.

7. Mention that the survey is closing on August 4th. It helps when people know that there’s a deadline and they need to act fast.

8. We’re donating $0.10 for every response to a charity that developers can select when they enter the survey. That’s a nice way to motivate people to support you and our survey too!

Hope these quick tips will inspire you and keep you in the game so that you can climb the leaderboard and win our top 50 prizes. Just remember to play fair and square and only promote the survey to real developers and software creators (sorry, you can’t ask your mom or cousin to click randomly!). Read our Referral Terms & conditions to make sure you’re playing by the rules. Good luck!

Categories
Tips

10 books on computer vision and cryptocurrency you should read

computer vision and cryptocurrency books

In our latest Developer Nation Pulse report we shared data on the top five emerging areas of interest to developers.

Around half of developers say they are working on, learning about, or interested computer vision, according to the insights from our Q1 2021 global survey of over 17,000 developers. Similarly, 45% are interested in cryptocurrencies (e.g. Bitcoin).

Recommended computer vision and cryptocurrency books

However, of the developers engaged with computer vision, only 15% are currently working on the technology. Similarly, only 14% are currently working on cryptocurrencies. One in four developers are currently learning about computer vision, while 29% are learning about cryptocurrencies.

So if you belong to these group, the following book recommendations might be just the thing you’ve been looking. This post was created in partnership with our friends at Packt.

Computer Vision books

Modern Computer Vision with PyTorch

Explore deep learning concepts and implement over 50 real-world image applications.

What reviews say:

“I felt the book is very well structured and compiled. Unless you’re looking for something very very specific, you’d be able to find techniques/implementations for any and all types of problems you are working on. They cover algorithms and implementations of basic neural networks, all the way upto RNNs and reinforcement learning with PyTorch. The breadth covered by this book on the number of techniques and algorithms is really amazing.”

Mastering Computer Vision with TensorFlow 2.x

Build advanced computer vision applications using machine learning and deep learning techniques

What reviews say:

“There are many books out there / but this book stands out – very clear explanation of codes and contents, lots of detailed explanations for object detection, classification, visual search, matching and training in cloud.”

PyTorch Computer Vision Cookbook

Over 70 recipes to master the art of computer vision with deep learning and PyTorch 1.x

“This book is good for beginners to learn about writing deep learning model in PyTorch. Book goes from basic linear model to processing videos in PyTorch and covers variety of use cases e.g. use of GANs, Style transfer project.”

Applied Deep Learning and Computer Vision for Self-Driving Cars

Build autonomous vehicles using deep neural networks and behavior-cloning techniques

What reviews say:

“This book is about how to apply deep learning knowledge to solve self-driving car problems. The technologies mainly focus on computer vision areas. It gives readers lots of code samples, which can help readers to understand the concept in each chapter.”

TensorFlow 2.0 Computer Vision Cookbook

Implement machine learning solutions to overcome various computer vision challenges

What reviews say:

“By far, this is one of the best books to understand how to apply deep learning in the field of computer vision. The concepts have been clearly explained. It covers almost everything from image classification, image segmentation, object detection, etc”

Raspberry Pi Computer Vision Programming, Second Edition

Design and implement computer vision applications with Raspberry Pi, OpenCV, and Python 3

What reviews say:

“This book was very helpful for me because it covers a wide variety of computer vision topics and offers lots of well thought out code examples using Python, opencv, matplotlib, numpy and other computer vision software. I followed his examples on my RPi and found that they helped me get the format and arguments of opencv commands correctly to include little things like commas, parenthesis, brackets, optional arguments and the like.”

Hands-On Image Generation with TensorFlow

A practical guide to generating images and videos using deep learning

What reviews say:

“The book is a great quickstart into representation with neural networks. (I also read it more deeply at times and it is great for that as well. I myself have experience with high-throughput large scale autoencoders with TensorFlow and building Facial Recognition applications. I appreciated this book a lot.)”

Cryptocurrencies books for developers

Practical Artificial Intelligence and Blockchain

A guide to converging blockchain and AI to build smart applications for new economies

What reviews say:

“Addressing such large topics as artificial intelligence and blockchain at best is a very serious endeavor. Whereas blockchain after a decade plus of existence has developed a useful understanding within its marketplace, that is not at all true of artificial intelligence, better just AI. AI is now well beyond 6 decades of existence as a topic and yet remains in an evolving state with much debate and speculation worldwide, especially over ethical and scope issues. So given that the reader of this book may be either one-of or some combination of a professional scientist, a developer or simply someone wanting to learn, then yes, Ganesh Prasad Kumble’s Practical Artificial Intelligence and Blockchain book is both a good and useful read.”

Blockchain Development for Finance Projects

Building next-generation financial applications using Ethereum, Hyperledger Fabric, and Stellar

What reviews say:

“This book is for developers who want to learn blocking technology by building financial applications. Kudos to the author on providing coding examples and following it with explanation. Overall it is a good book on Ethereum development and I would recommend it for anyone who wants to learn Ethereum blockchain by building fintech applications.”

Securing Blockchain Networks like Ethereum and Hyperledger Fabric

Learn advanced security configurations and design principles to safeguard Blockchain networks

What reviews say:

“This book is for blockchain developers, security professionals, and Ethereum and Hyperledger developers who are looking to implement security in blockchain platforms and ensure secure data management using an example-driven approach. Basic knowledge of blockchain concepts will be beneficial.”

Is there a book or expert that you would recommend to others interested in cryptocurrency or computer vision? Do share in the comments.

Our latest developer survey is live. Let us know which emerging technology you’ll be exploring in 2021.

Categories
Tips

Hello, world. Meet Developer Nation!

developer nation community, developer nation survey

It’s a new era for our community, Developer Economics is now Developer Nation.

Over the last ten years we’ve gathered over 40,000 developers with one thing in common: an unyielding curiosity about what the future holds for software builders. Together with all of you we’ve run 20 global developer surveys, co-created over 300 blog posts, read over 20 reports, pondered over 300 graphs and tried to make sense of the emerging trends in the developer world and what they mean for you and your projects.

Now it’s time to reveal a new chapter. We want to be more than just a survey, a report or a newsletter you interact with from time to time. It’s time to shift the focus from us to you – a diverse group of curious and ambitions software creators who want more from the ecosystem: more support from the platforms, more value for your time, and more chances to learn from each other.

Why Developer Nation?

For the last 10 years we were known as Developer Economics. It helped us earned the trust of tens of thousands of developers over the years. But it’s not helping us resonate with many, many more software creators out there. The “Economics” part especially. We have evolved beyond analysing revenue models for developers. We’ve grown by mapping trends across a wider ecosystem of developer tools, platforms, technologies, and programs. The future is about helping all software creators (developers and no-coders alike) influence how the software is built.

We approached the new name from different angles: blending concepts, metaphors, acronyms, dev jargon, pop culture, creative spelling, crunchy suffixes, Latin words, fringe Greek gods, and more. We came up with 157 name ideas! We then asked around 1,000 of our most loyal members for a feedback. Developer Nation is the name that stood out the most with you. It was associated with trust and excitement.

With a new name comes a new responsibility

Our vision is a software development ecosystem that listens, empowers, and supports software creators to pursue continuous learning, build future-forward solutions without compromises. We want to help you feel confident in your technology choices and included in the conversation about the future of the ecosystem.

We pledge to do this through a global research, sharing with you trustworthy data about emerging software trends, and offering you new opportunities to connect with the people at the service of software builders.

How can you get more involved?

We’ll be testing new programs and channels through which you can connect with other software creators and platform leaders to exchange ideas and experiences.

We want to give you more space and opportunities to speak about the topics that you find relevant and other software builders will care about:

  • What’s in your toolbox?
  • What was your journey to becoming a software creator like? Your biggest mistakes and “a-ha!” moments?
  • Which software development trends need to be questioned and which deserve more attention, in your expert opinion?
  • Which books / podcasts / events / blogs / people inspire you?
  • What projects are you working on and how can our community help you test, give feedback, and earn users or followers?

Find out how you can get involved here.

We’ll still invite you to participate in our global surveys twice a year.

We’d also love for you to read our blog, reports, and newsletter.

It’s a feature, not a bug

Some of our links and form still refer to developereconomics.com and your user profile is still a bit of work-in-progress. Hang in there, we’ll get it sorted!

Say hello to our team!

Last but not least, it’s about time you met the people behind the screens who want to turn this into a real community!

Categories
Tools

AngularJS development tools for 2021

AngularJS development tools

Business and economy took the worst hit during the 2020 pandemic and the situation is making a comeback again this year. The adversities also paved the way to recognize the digital approach towards business. While digital marketing, websites, social media prevailed and was put to good use by businesses even before this situation, the pandemic made it a necessity rather than an option.

Businesses joined the online race to boost their company and stay afloat in these difficult times. All these together showed us how a business model can be re-thought based entirely on the internet.

The shift to e-commerce specifically called for simpler technologies for businesses as well as users and as a result, exponential growth in the requirement for web development tools has been witnessed. 

One such option is the AngularJS development tools. It is an open-source application that is particularly useful for making single-page applications. Angular JS Development Companies are significantly using the AngularJS Frameworks to simplify their works. 

We have listed some of the top AngularJS Development tools which are trending in 2021 here to give you an insight into the best options you can look out for.

5 Trending AngularJS development tools in 2021

With the help of AngularJS development tools, you can easily create a platform to promote your business—from a simple interface to comparatively complex applications. The following are top trending AngularJS platforms that are growing in popularity by the day:

Karma

This is one of the most popular testing frameworks for AngularJS Development. The primary perks of using this one are the option to test the application on a browser as well as a different device like tablets or smartphones. Unit testing, E2E testing, and midway testing are primarily conducted in this.

Features of Karma AngularJS Framework:

  • Versatile and can be used on all browsers
  • Open-source platform
  • Can be easily integrated with Travis, Jenkins, and Semaphore
  • Debugging is made easier
  • Integration is smooth

Pros and cons:

  • You can test it on real devices.
  • There are several plug-ins available to make your task simpler.
  • However, it is a bit slow compared to many other such tools.

Jasmine

If you want to test Javascript Codes, the Jasmine AngularJS Development tool is the one for you. It is driven by a behaviour approach and an independent framework that is not reliant on DOM javascript. 

Projects based on NodeJS, particularly find this useful, and any other project where the javascript works will also benefit from this. Accenture, Gitlab, Walmart are a few of the corporations among the 200 others which are built using Jasmine.

Features of Jasmine are as follows:

  • It is a fast framework with lower overhead.
  • There is an included library that you can access while using this framework.
  • It can be used for both browser and Node.
  • It does not require DOM.

Pros and Cons:

  • It is an open-source framework with a set of straightforward syntax for easier testing.
  • Coding is easier here and you also get community support along with thorough documentation.
  • The front end, as well as the backend test, are both simplified.
  • However, asynchronous testing can be a little bit tricky.

AngularFire

AngularFire is known as one of the best backend development tools. Dynamic API features and 3-way data binding can easily be facilitated with this tool. When you need a rapid development process, this is going to work. The most attractive feature of AngularFire is its ability to synchronize real-time binding and save the data in automated mode.

Features of AngularFire that make it so popular:

  • Data Storage is automatic.
  • Data syncing is done in real-time.
  • Binary files like videos and images are managed easily.
  • Firebase services are easily projected and injected.

Pros and Cons:

  • High security for the Angular roots and flexible and dynamic API.
  • The developer experience is simple and effective.
  • Documentation features are not considered up to the mark by many developers which is the only drawback of this AngularJS tool.

Djangular

It comes with several functionalities that help to focus on the primary development process and based on a philosophy called DRY or Don’t Repeat Yourself. Many leading e-commerce websites and social companies adopt this tool because it has a robust content delivery network.

Scalability, security, and rapid development are assured by this AngularJS development tool.

Features of Djangular:

  • You can get namespacing AngularJS content.
  • The frameworks are very flexible.

Pros and Cons:

  • It is protected by CSRF Security.
  • It is a reusable app and you can develop project-specific content.
  • Though an open-source development tool, it is a bit too advanced for beginners.

Mocha JS

Another popular JavaScript testing framework, Mocha JS, is known for flexibility and overall precise reporting. Many inbuilt features are incorporated in this tool and can also run on Node.JS. It sanctions asynchronous testing with mapping and broadcasting.

It also has global leak detection capabilities, browsing support, asynchronous support, and much more. In short, the Mocha.JS is ideal for producing suites and custom projects using its functionalities.

Features of Mocha.JS:

  • The framework has several features.
  • You get browser and async support.
  • Auto exit and testing support are also available with Mocha.JS.
  • There is a JavaScript API for testing.
  • The ease of set-up of this AngularJS tool is unbeatable.

Pros and Cons:

  • There is optimum test reporting and offers much more flexibility.
  • The multiple browser compatibility is commendable.
  • You get both TDD and BDD or Test Driven Development and Behavior Driven Development.

When you are trying to develop an app or website for your business, there are so many different features and aspects to keep in mind. This calls for using the right development tools. By opting for any of the above-mentioned tools, you can fulfill the requirement thoroughly. 

Now that you know the different popular AngularJS development tools and what their features are, you can determine easily how to put any of these to good use. Boosting your entrepreneurial ventures even in these difficult times is made simpler when you embrace the right technology.

About

Harikrishna Kundariya, is a marketer, developer, IoT, ChatBot & Blockchain savvy, designer, co-founder, Director of eSparkBiz Technologies. His 8+ experience enables him to provide digital solutions to new start-ups based on IoT and ChatBot.

Categories
Analysis

Infographic: Programming languages adoption trends 2021

Infographic: Programming Language adoption trends 2021

In our last infographic, JavaScript was the most popular programming language. What has changed in terms of the sizes in the last six months? You can find the answers in this infographic with key findings from our Developer Economics 20th edition survey, which ran between November 2020 and February 2021 and reached 19,000 developers worldwide.

Javascript is the queen of programming languages

JavaScript is the most popular programming language by some distance, with nearly 14M developers using it globally. More importantly, the JavaScript community has been growing in size consistently for the past three years. Between Q4 2017 and Q1 2021, more than 4.5M developers joined the community – the highest growth in absolute terms across all languages. Even in software sectors where JavaScript is not among developers’ top choices, like data science or embedded development, about a fourth of developers use it in their projects.

Python is conquering the world

Since it surpassed Java in popularity at the beginning of 2020, Python has remained the second most widely adopted language behind JavaScript. Python now counts just over 10M users, after adding 1.6M net new developers in the past year alone. That’s a 20% growth rate, the highest across all the large programming language communities of more than 6M users. The rise of data science and machine learning (ML) is a clear factor in Python’s popularity. Close to 70% of ML developers and data scientists report using Python. For perspective, only 17% use R, the other language often associated with data science.

Kotlin’s rise continues

The fastest growing language community in percentage terms is Kotlin. In fact, it’s one of the two communities – the other being Rust – that has grown more than two-fold over the last three years, from 1.1M developers in Q4 2017 to 2.6M in Q1 2021. This is also very

evident from Kotlin’s ranking, where it moved from 11th to eight place during that period – a trend that’s largely attributed to Google’s decision to make Kotlin its preferred language for Android development. Even so, Kotlin still has a long way to go to catch up with the leading language in mobile development, Java; there are currently twice as many mobile developers building applications in Java than in Kotlin.

Swift was recently outranked by Kotlin, after attracting slightly fewer net new developers in the second half of 2020 (100K vs 300K). Even so, Swift is currently the default language for development across all Apple platforms, which has led to a stagnation in the adoption of Objective C. This gradual phase-out of Objective C from the Apple app ecosystem is also matched by a significant drop in its rank, from ninth to 12th place. 

The more niche languages – Go, Ruby, Rust, and Lua – are still much smaller, with up to 2.1M active software developers each. Go and Ruby are important languages in backend development, but Go has grown slightly faster in the past year, both in absolute and percentage terms. Rust has formed a very strong community of developers who care about performance, memory safety, and security. As a result, it grew faster than any other language in the last 12 months, more than doubling in size. Finally, Lua was also among the fastest growing language communities in the last year, mainly attracting AR/VR and IoT developers looking for a scripting alternative to low-level languages such as C and C++.

Sign up to our community to have your say in our next developer survey.

Infographic: Programming languages adoption trends 2021
Categories
Analysis

The search for a cloud-native database

cloud-native database

Cedrick Lunven (@clunven) and Jeff Carpenter (@jscarp) of K8ssandra discuss the search fora cloud-native database.

The concept of “cloud-native” has come to stand for a collection of best practices for application logic and infrastructure, including databases. However, many of the databases supporting our applications have been around for decades, before the cloud or cloud-native was a thing. The data gravity associated with these legacy solutions has limited our ability to move applications and workloads. As we move to the cloud, how do we evolve our data storage approach? Do we need a cloud-native database? What would it even mean for a database to be cloud-native? Let’s take a look at these questions.

What is Cloud-Native?

It’s helpful to start by defining terms. In unpacking “cloud-native”, let’s start with the word “native”. For individuals, the word may evoke thoughts of your first language, or your country or origin – things that feel natural to you. Or in nature itself, we might consider the native habitats inhabited by wildlife, and how each species is adapted to its environment. We can use this as a basis to understand the meaning of cloud-native.

Here’s how the Cloud Native Computing Foundation (CNCF) defines the term:

“Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds: Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach.

These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.”

This is a rich definition, but it can be a challenge to use this to define what a cloud-native database is, as evidenced by the Database section of the CNCF Landscape Map:

Database section of the CNCF Landscape Map

Databases are just a small portion of a crowded cloud computing landscape.

Look closely, and you’ll notice a wide range of offerings: both traditional relational databases and NoSQL databases, supporting a variety of different data models including key/value, document, and graph. You’ll also find technologies that layer clustering, querying or schema management capabilities on top of existing databases. And this doesn’t even consider related categories in the CNCF landscape such as Streaming and Messaging for data movement, or Cloud Native Storage for persistence.

Which of these databases are cloud-native? Only those that are designed for the cloud, should we include those that can be adapted to work in the cloud? Bill Wilder provides an interesting perspective in his 2012 book, “Cloud Architecture Patterns”, defining “cloud-native” as:

“Any application that was architected to take full advantage of cloud platforms”

By this definition, cloud-native databases are those that have been architected to take full advantage of underlying cloud infrastructure. Obvious? Maybe. Contentious? Probably…

Why should I care if my database is cloud-native?

Or to ask a different way, what are the advantages of a cloud-native database? Consider the two main factors driving the popularity of the cloud: cost and time-to-market.

  • Cost – the ability to pay-as-you-go has been vital in increasing cloud adoption. (But that doesn’t mean that cloud is cheap or that cost management is always straightforward.)
  • Time-to-market – the ability to quickly spin up infrastructure to prototype, develop, test, and deliver new applications and features. (But that doesn’t mean that cloud development and operations are easy.)

These goals apply to your database selection, just as they do to any other part of your stack.

What are the characteristics of a cloud-native database?

Now we can revisit the CNCF definition and extract characteristics of a cloud-native database that will help achieve our cost and time-to-market goals:

  • Scalability – the system must be able to add capacity dynamically to absorb additional workload
  • Elasticity – it must also be able to scale back down, so that you only pay for the resources you need
  • Resiliency – the system must survive failures without losing your data
  • Observability – tracking your activity, but also health checking and handling failovers
  • Automation – implementing operations tasks as repeatable logic to reduce the possibility of error. This characteristic is the most difficult to achieve, but is essential to achieve a high delivery tempo at scale

Cloud-native databases are designed to embody these characteristics, which distinguish them from “cloud-ready” databases, that is, those that can be deployed to the cloud with some adaptation.

What’s a good example of a cloud-native database?

Let’s test this definition of a cloud-native database by applying it to Apache Cassandra™ as an example. While the term “cloud-native” was not yet widespread when Cassandra was developed, it bears many of the same architectural influences, since it was inspired by public cloud infrastructure such as Amazon’s Dynamo Paper and Google’s BigTable. Because of this lineage, Cassandra embodies the principles outlined above:

  • Cassandra demonstrates horizontal scalability through adding nodes, and can be scaled down elastically to free resources outside of peak load periods
  • By default, Cassandra is an AP system, that is, it prioritizes availability and partition tolerance over consistency, as described in the CAP theorem. Cassandra’s built in replication, shared-nothing architecture and self-healing features help guarantee resiliency.
  • Cassandra nodes expose logging, metrics, and query tracing, which enable observability
  • Automation is the most challenging aspect for Cassandra, as typical for databases.

While automating the initial deployment of a Cassandra cluster is a relatively simple task, other tasks such as scaling up and down or upgrading can be time-consuming and difficult to automate. After all, even single-node database operations can be challenging, as many a DBA can testify. Fortunately, the K8ssandra project provides best practices for deploying Cassandra on Kubernetes, including major strides forward in automating “day 2” operations.

Does a cloud-native database have to run on Kubernetes?

Speaking of Kubernetes… When we talk about databases in the cloud, we’re really talking about stateful workloads requiring some kind of storage. But in the cloud world, stateful is painful. Data gravity is a real challenge – data may be hard to move due to regulations and laws, and the cost can get quite expensive. This results in a premium on keeping applications close to their data.

The challenges only increase when we begin deploying containerized applications using Kubernetes, since it was not originally designed for stateful workloads. There’s an emerging push toward deploying databases to run on Kubernetes as well, in order to maximize development and operational efficiencies by running the entire stack on a single platform. What additional requirements does Kubernetes put on a cloud-native database?

Containerization

First, the database must run in containers. This may sound obvious, but some work is required. Storage must be externalized, the memory and other computing resources must be tuned appropriately, and the application logs and metrics must be made available to infrastructure for monitoring and log aggregation.

Storage

Next, we need to map the database’s storage needs onto Kubernetes constructs. At a minimum, each database node will make a persistent volume claim that Kubernetes can use to allocate a storage volume with appropriate capacity and I/O characteristics. Databases are typically deployed using Kubernetes Stateful Sets, which help manage the mapping of storage volumes to pods and maintain consistent, predictable, identity.

Automated Operations

Finally, we need tooling to manage and automate database operations, including installation and maintenance. This is typically implemented via the Kubernetes operator pattern. Operators are basically control loops that observe the state of Kubernetes resources and take actions to help achieve a desired state. In this way they are similar to Kubernetes built-in controllers, but with the key difference that they understand domain-specific state and thus help Kubernetes make better decisions.

For example, the K8ssandra project uses cass-operator, which defines a Kubernetes custom resource (CRD) called “CassandraDatacenter” to describe the desired state of each top-level failure domain of a Cassandra cluster. This provides a level of abstraction higher than dealing with Stateful Sets or individual pods.

Kubernetes database operators typically help to answer questions like:

  • What happens during failovers? (pods, disks, networks)
  • What happens when you scale out? (pod rescheduling)
  • How are backups performed?
  • How do we effectively detect and prevent failure?
  • How is software upgraded? (rolling restarts)

Conclusion and what’s next

A cloud-native database is one that is designed with cloud-native principles in mind, including scalability, elasticity, resiliency, observability, and automation. As we’ve seen with Cassandra, automation is often the final milestone to be achieved, but running databases in Kubernetes can actually help us progress toward this goal of automation.

What’s next in the maturation of cloud-native databases? We’d love to hear your input as we continue to invent the future of this technology together.

This blog post originally appeared on K8ssandra and is based on Cedrick’s presentation “Databases in the Cloud-Native Era” from BluePrint London, March 11, 2021 (registration required).

Categories
Tips

7 DevOps books to read in 2021

7 DevOps books to read in 2021

If you are looking to learn more about Ansible, Azure, Docker, Terraform, Kubernetes, and their roles in DevOps, then this blog post is for you. We continue our series of must-read books with 7 DevOps books to read in 2021, as recommended by our friends at Packt.

Azure DevOps Explained

Get started with Azure DevOps and develop your DevOps practices

What reviews say:

” In my opinion, it is definitely one of the greatest books I ever read for DevOps.
Although I am Azure DevOps certified, I really enjoy reading this book and it gives me an extra overview of what I have learned.
It is well structured and the fact that is simple to read and follow along makes it more attractive. “

Terraform Cookbook

Efficiently define, launch, and manage Infrastructure as Code across various cloud platforms

What reviews say:

” I had the chance to read this book and I was really pleased by its content.
noting that this is not the first book or terraform material that I read, I would say that this book contains valuable structured information with also access to code used in various chapters.
it is certainly an asset for those starting their journey with terraform.”

Practical Ansible 2

Automate infrastructure, manage configuration, and deploy applications with Ansible 2.9

What reviews say:

“This book is probably perfect for someone with reasonable experience. It was what I needed as a second book to get a good look at the ecosystem and a second opinion of how to use it. “

Kubernetes – A Complete DevOps Cookbook

Build and manage your applications, orchestrate containers, and deploy cloud-native services

What reviews say:

” Great coverage of common Kubernetes and DevOps tools. I’ve learned about some of the tools I haven’t used before like Jenkins X, GitLab, Fossa, Trivy, Litmus Chaos etc.
Although some of the long YAML files are provided in the GitHub repository I got the digital version, makes it easier to copy paste. “

Kubernetes and Docker – An Enterprise Guide

Effectively containerize applications, integrate enterprise systems, and scale applications in your enterprise

What reviews say:

“If you have worked on Kubernetes at all, you have experienced the frustration of trying to go beyond a cluster that has a single config file and a simple layer 7 load-balancer using NGINX. This book does truly target not only the enterprise user, but any person that wants to learn topics that make Kubernetes a complete offering.

I have been looking into the external-dns project on my list for a few months, but I never got around to doing much – Much to my surprise, when I was reading the topics covered in the book, it mentioned Services and external-dns. Chapter 6, alone, to me is one reason to buy the book since it explained and showed me how to install Metallb with external-dns in easy to understand terms and hands-on configuration.”

Learning DevOps

The complete guide to accelerate collaboration with Jenkins, Kubernetes, Terraform and Azure DevOps

” I would suggest reading through each section before you work along with the steps. There’s lots of references to other resources that are not necessarily part of the topics being discussed ”

Docker for Developers

Develop and run your application with Docker containers using DevOps tools for continuous delivery

” When reading articles, tutorials and even books, that is very common that at the end of the reading you struggle about how to translate that to a real production situation. Believe me, this book is different. You get to the end with a sense that you are very likely to know what are the next steps to apply what you learned to your existent or new projects. And this means a lot. The book has some great balance from history, concepts, example and practice. ”

What books have helped you deepen your knowledge of DevOps? Do share in the comments. Looking for more books to read? We have also shared recommended Backend and Frontend books.