Introduction​

Task Scheduling is one of the most important features in software systems, which literally means assigning and executing long tasks or scripts according to certain specifications. In the web crawler management platform Crawlab, task scheduling serves as a core module, which you may wonder how to build it from scratch. This article will introduce you how to build a simple but useful task scheduler with Go.

Idea​

Let's focus on what we need for the task scheduling system.

• User Interface: API
• Scheduler: Cron
• Execute Tasks: Executor

Below is the basic process.

We can use HTTP API to create scheduled tasks, and the executor will execute scripts periodically based on their specifications.

Introduction​

"You'd better stay late tonight. Our boss is angry about the progress." -- An ambitious but worrying project manager

Agile, a word once standing for agility and speed, has become one of the sexiest but seemingly abstruse jargons often mentioned by software developers in the IT industry over past 20 years. Agile Development is the primary option of project management methodologies for many teams, because it is simple, agile (literally), and more importantly implying fast. Think about countless programming labors who are willing to work overtime voluntarily because of hard deadlines.

Agile = Deliver Fast?​

When your dev team suffers deferred releases, product quality degradation, poor team moral and fading customer relationship, your friends probably will recommend you using agile development, "Hey, buddy. I heard that agile is pretty awesome. You should probably take a shot." However, after your team has been practicing agile for a while, you are very likely to find it not good enough: loads of bugs after production releases, increasing number of overtime hours, and endless new requirements each of which is the highest priority. You are trembling with fear in the regular meeting with your boss who may ask, "I heard that you guys are improving our capability of delivering with Agile. This is great! So can those important features be available next week?"

Agile is the same as delivering fast, this is one of the biggest myths from many agile practitioners. Literally, Agile doesn't necessarily mean fast; instead, introduction of Agile will lead to more work and therefore lower delivering speed. Yeah, you are right. Technically speaking, Agile will reduce the speed of delivery. For example, Extreme Programming (XP, one of the most popular agile frameworks) requires unit test cases to cover all features and functions, which would result in 1-2 times of extra code.

So, why do people claim that Agile works great in software development? As it can neither increase development efficiency, nor does it reduce workload, what on earth is this thing useful? This is a typical and the most critical question for Agile practitioners. If you cannot provide timely answers that is satisfied to the team, they will ultimately drop Agile and head back to their old way.

Preface​

Today's heroes in this world are only you and me. -- Cao Cao, "Romance of the Three Kingdoms"

For programmers who have been working in the IT industry for many years, if asked about the most mainstream backend programming language in China, I believe most would say Java. This isn't surprising, as Java has existed for over 30 years, has a massive user base and ecosystem, and seems to hold an absolute dominant position in the software engineering field. However, as ancient wisdom says: "Those who win the hearts of the people win the world." The programming language with the most users isn't necessarily the most beloved by developers. According to StackOverflow's 2021 survey of 82,914 developers on programming language satisfaction, only 47% liked Java, ranking it beyond 20th place, barely higher than PHP, C, and COBOL. On the other hand, we can see from the survey results that C#, often called the "knockoff Java," actually achieved 62% satisfaction among developers, 15% higher than Java. Although C#'s satisfaction still has a considerable gap compared to Rust and TypeScript, we can see that C# as an alternative programming language to Java is gradually gaining advantages in development efficiency, deployment convenience, and documentation completeness. Due to work requirements, I've used both C# and Java to develop numerous projects, giving me some understanding of their similarities, differences, advantages, and disadvantages. I believe there are good reasons why C# is more popular among developers than Java - it offers a great development experience.

Due to length constraints, the complete introduction to C# principles and practical applications (i.e., why I recommend using C# to build large-scale backend applications) will be split into a series of articles. This series will deeply analyze this "young" programming language from dimensions such as syntax features, development patterns, ecosystem, deployment and building, using the cross-platform framework .NET Core as an example to introduce how to build large-scale backend applications with C#.

This article is the first in the C# series, primarily introducing some modern syntax features of C# and how they improve development efficiency.

Introduction​

A bad plan is better than no plan.

坏计划也好过没有计划。--彼得·蒂尔《从0到1》

In software development engineering, there are rarely lone wolf programmers. This is because modern, commonly seen software projects are usually very complex, requiring substantial human resources, resources, and time. Having a single developer complete a large software project alone would be like "an old man moving mountains." Therefore, software development is inseparable from team collaboration and project management. Project Management, simply put, is a methodology for orderly organizing, planning, executing, and completing various tasks in a project. Of course, the actual scope of project management goes far beyond this, usually involving resource allocation, priority setting, progress tracking, etc. It's a product of the Industrial Revolution and a branch of modern management science that can significantly improve engineering completion efficiency and success rates. This article mainly discusses software project management, which is very different from traditional project management in construction engineering, mechanical engineering, etc. Early IT project management borrowed from traditional project management methodologies like construction engineering, playing an important role in the early information age and significantly improving software development and collaboration efficiency. However, with the rapid development of the IT industry, consumer product demands change rapidly, and market conditions have become increasingly volatile. Traditional software project management models can no longer meet software development needs. Therefore, modern software development models, such as Agile Development, emerged and became the preferred choice for many internet companies.

What are the drawbacks of traditional project management models (such as waterfall)? What improvements do modern project management models (such as agile) offer? Should we completely abandon waterfall models and fully embrace agile development? As a programmer, should you master some project management knowledge and related tools? As a team leader, how should you establish project management processes to ensure development efficiency and quality? If readers have similar questions, this article will provide detailed analysis and answers.

Introduction​

Those outside the city want to get in, those inside the city want to get out. -- Qian Zhongshu "Fortress Besieged"

With the continuous popularity of Container Orchestration, Microservices, Cloud Technology and other trends in the IT industry, Golang (Go language, abbreviated as Go), born at Google in 2009, is increasingly welcomed and sought after by software engineers, becoming today's hottest backend programming language. In the list of software projects developed with Golang, there are star-level products like Docker (container technology) and Kubernetes (container orchestration) that have disrupted the entire IT industry, as well as powerful and practical well-known projects like Prometheus (monitoring system), Etcd (distributed storage), and InfluxDB (time-series database). Of course, Go language's application domains are by no means limited to containers and distributed systems. Today, many large internet companies are extensively using Golang to build backend Web applications, such as Toutiao, JD.com, Qiniu Cloud, etc. The web scraping field, long dominated by Python, is also being continuously challenged by Golang due to the rise of the simple and easy-to-use scraping framework Colly. Golang has become the programming language that most software engineers want to learn today. The image below shows relevant results from HackerRank's 2020 programmer skills survey.

So, is Go language really a lifesaver for backend developers? Can it effectively improve programmers' technical capabilities and development efficiency, thus helping them advance further in their careers? Is Go language really worth spending a lot of time learning in depth? This article will provide detailed introduction to Golang's language characteristics, its advantages and disadvantages, and applicable scenarios. With the above questions in mind, it will analyze various aspects of Go language to help programmers new to the IT industry and developers interested in Go to further understand this popular language.

Preface​

The most incomprehensible thing about the world is that it is comprehensible.

The world's most incomprehensible aspect is that it is comprehensible. -- Albert Einstein

Open-source has created today's thriving software industry. Open-source enables developers worldwide to collaborate on excellent utility projects, also known as "wheels," benefiting companies and individuals of all sizes while showcasing creators' and contributors' technical prowess. Many developers today extensively use open-source projects as third-party libraries or dependencies to complete development tasks faster and more efficiently.

I'm no exception. Recently, while refactoring the Crawlab frontend using Vue 3, I used the upgraded version of ElementUI developed by the Element team - the new UI framework Element Plus rebuilt for Vue 3. The Element team completely refactored Element Plus using Vue 3, fully embracing TypeScript. Compared to the previous Vue 2 version, it enriched some components while maintaining consistent styling and usage patterns. Some APIs even became more streamlined. Therefore, during the initial refactoring phase of Crawlab's frontend, I didn't encounter major obstacles. Combined with previous development experience, the development process felt smooth and familiar. However, the good times didn't last long. As the project continued developing, I encountered some technical difficulties. More precisely, I faced limitations from the Element Plus framework itself when implementing complex features. Although I eventually found ways to solve the problems, I deeply experienced the difficulties of hacking open-source project source code. Therefore, I'd like to take this opportunity to share my experience in mastering open-source code with readers.

This article will begin with my experience solving Element Plus problems, progressively discussing issues with open-source projects or frameworks, and further discussing methods and techniques for mastering open-source project source code, sharing thoughts on reading, understanding, and modifying source code. This article primarily focuses on methodology discussions without too many technical details, making it accessible to readers of any professional background.

Introduction​

The only constant in the world is change.

世界上唯一不变的是变化。--《谁动了我的奶酪》作者 斯宾塞·约翰逊

The IT industry changes too fast, especially frontend development. If you could travel back 10 years and meet a Web development software engineer, they would definitely tell you that mastering frontend means being proficient with jQuery and solving IE browser compatibility issues. However, with the continuous development of frontend, jQuery faced "official retirement" and gradually exited the historical stage (element selection and manipulation were unified by standard DOM APIs); the much-criticized IE browser compatibility problems, due to IE's gradually shrinking market and the emergence of compatibility tools (Polyfill), have been downgraded from core optimization issues to minor annoyances, no longer being standard equipment for frontend engineers.

Today's frontend development has a dazzling array of professional terminology and complex technology ecosystems, which may make engineers new to frontend development feel anxious: there's simply too much to learn. Modern frontend engineers who don't understand Webpack, Babel, Node.js, NPM/Yarn, ES6/7, React/Vue, Sass/Less, TypeScript, ESLint, Canvas/SVG and other modern frontend knowledge will find it difficult to convince others of their professional background. Frontend engineers in 2021 may be true engineers in every sense of the word. They typically need to apply extensive professional knowledge to solve engineering problems, including how to modularize projects, how to design interactions between components, how to improve reusability, how to enhance bundling efficiency, optimize browser rendering performance, etc. They no longer just need the HTML/CSS/JS routine to develop static pages like before.

This article will focus on the theme of modern frontend development to introduce various important technologies of frontend engineering in detail, helping readers understand how the complexity and diversity of modern frontend pages are constructed. This article is a popular science piece about frontend engineering—even if you don't understand frontend technology, you can benefit from this article.

Introduction​

A craftsman must first sharpen his tools if he is to do his work well. -- The Analects of Confucius

In today's frontend landscape dominated by three major frameworks, very few people don't know Vue. In 2014, former Google engineer Evan You released what's called a progressive frontend application framework, Vue. Its simplified template binding and component-based thinking had a positive and profound impact on the frontend field that was still in the jQuery era. The birth of Vue benefited frontend developers who weren't comfortable with TS or JSX syntax. Moreover, Vue's low learning threshold also makes it very easy for beginners to get started. This is also an important reason why Vue could spread rapidly in a short time. From the State of JS survey, we can see that Vue's awareness is close to 100%, and overall user satisfaction is also quite high.

Vue is both powerful and easy to learn—does this mean Vue is a perfect framework? Unfortunately, the answer is no. Although Vue has a low entry threshold and is flexible and easy to use, this advantage also becomes a double-edged sword, bringing certain limitations for building large projects. Many frontend engineers who have developed large projects with Vue 2 have a love-hate relationship with Vue. However, with the release of Vue 3, these disadvantages that became prominent when developing large projects have been effectively resolved, making the Vue framework very versatile and truly having the potential to compete with "frontend framework leader" React. What important new features does Vue 3 bring? This article will provide a detailed introduction.

Introduction​

How can I blame you for making mistakes, when I gave you too much freedom. -- Jacky Cheung "Too Much"

Many software engineers probably know or understand TypeScript (abbreviated as TS) to some extent, and frontend developers who have used TypeScript consistently express positive feelings about it. If you search for TypeScript on search engines, you'll find an overwhelming number of articles praising or complimenting TS, such as "TypeScript Makes You Never Want to Go Back to JavaScript", "TypeScript Sweet Series", and "If You Don't Embrace TypeScript, You're Getting Old!". According to the latest 2020 State of JS Survey Report, TypeScript's popularity and satisfaction are increasing year by year, with "TS fans" (developers who answered "would continue using" in State of JS) including this author, even exceeding 70% (as shown below).

In summary, TS now holds an unshakeable core position in the frontend field and is a very important frontend engineering development tool. TS is one of Microsoft's outstanding contributions to the software industry after embracing open source projects. However, TypeScript cannot improve the runtime efficiency of JavaScript code in browsers, nor can it increase developer productivity like React and Vue frontend frameworks, and it certainly cannot make your frontend pages look attractive. So what exactly makes it such a widely beloved "truly fragrant language"? What makes frontend developers love it so much? If you have similar questions, please continue reading. This article will explain in detail the advantages of using TS to develop large-scale frontend projects.

Background​

Many excellent programmers and technical professionals enjoy writing technical articles and blogs as a way to share and spread knowledge and experience, expand their visibility and influence, attract followers, and some tech bloggers even earn advertising revenue through writing. Many excellent bloggers have also gained opportunities to publish books and find jobs through this method. Therefore, writing technical articles is a very worthwhile investment that helps oneself while benefiting the public.

However, writing technical articles is usually time-consuming, especially for high-quality articles. Not only do they require extensive research, thoughtful article structure, and consideration of the target audience, but they also require a lot of preliminary work, such as setting up environments, writing demo code, testing code, and so on. A high-quality technical article typically takes 3-6 hours to complete. However, spending a lot of time writing articles only to have them receive little attention upon publication is quite frustrating. We believe that quality articles deserve attention and dissemination, allowing more technical workers to gain knowledge and benefit from reading them.

Every tech blogger has their preferred technical media platforms, such as Juejin, CSDN, WeChat Official Accounts, etc. Many tech bloggers also like to publish articles on different platforms to seek maximum attention while preventing their hard work from being copied and plagiarized by others. However, publishing on multiple platforms is troublesome: bloggers need to log into multiple media platforms simultaneously and copy-paste their articles one by one. What's more troublesome is that some platforms only support Markdown while others only support rich text, requiring bloggers to convert between the two, which increases workload.

The multi-platform publishing tool ArtiPub solves exactly this problem. Below, we'll introduce the recently launched open-source multi-platform publishing platform ArtiPub.