Meta is hard and OpenAI, and the domestic "small model" official announces open source. Where is the "Hundred Models War" going?

  Since the beginning of this year, the global Internet giants have set off a "hundred-model war", and Microsoft, Google, Baidu and Ali have come to the end one after another. After more than half a year of competition, technology giants are welcoming a new round of road disputes around the big model ecology: facing the parameter "ceiling", will the future of the big model be closed or open?

  The open source model can run on a home computer.

  On August 3rd, two open source models, Qwen-7B and Qwen-7B-Chat, were put on the domestic AI developer community "ModelScope", which were Alibaba Cloud Tongyi Qianwen’s 7 billion parameter general model and dialogue model respectively. Both models were open source, free and commercially available.

  According to reports, Tongyi Qianwen Qwen-7B is a pedestal model that supports many languages such as Chinese and English, and it is trained on more than 2 trillion token (text unit) data sets, while Qwen-7B-Chat is a Chinese-English dialogue model based on the pedestal model, which has reached the cognitive level of human beings.In short, the former is like a "foundation" and the latter is a "house" on the foundation.

  The actual test shows that the comprehensive performance of Qwen-7B model is good. Among them, on the English proficiency evaluation benchmark MMLU, the score is generally higher than that of the mainstream models with the same parameter scale, even surpassing some models with 12 billion and 13 billion parameter scales. On the Chinese evaluation C-Eval verification set, the model also achieved the highest score of the same scale. Qwen-7B model is also among the best in evaluating GSM8K in mathematical problem solving ability and HumanEval in code ability.

  That is to say,In the tests of Chinese and English writing, solving mathematical problems and writing codes, Qwen-7B model is properly a "master of learning", and its score even exceeds the international mainstream model with the same parameter level.

  Besides, the industry is more concerned about the usability of Qwen-7B model. As we all know, the training and operation of mainstream large models need special AI training chips (such as NVIDIA A100), which are not only expensive, but also as high as 10,000 — per NVIDIA A100; 15,000 dollars, and it is monopolized by countries such as Europe and the United States, and it is almost impossible to buy it in China.The domestic Qwen-7B model supports the deployment of consumer graphics cards, which is equivalent to a high-performance home computer to run the model.

  Thanks to free commercialization and low threshold, the Qwen-7B model has been put on the shelves, which has attracted the attention of AI developers.In just one day, on the code hosting platform GitHub, the Qwen-7B model has been collected by more than a thousand developers, and most of the questioners are Chinese developers.As Alibaba Cloud said in the statement: "Compared with the lively AI open source ecology in the English-speaking world, the Chinese community lacks an excellent pedestal model. The addition of Tongyi Qianwen is expected to provide more choices for the open source community and promote the open source ecological construction of AI in China. "

  Open source or closed?

  In fact, Qwen-7B model is not the first big open source model. In fact, GPT-2, the predecessor of ChatGPT, is also completely open source. Its code and framework can be used for free on the Internet, and related papers can be consulted. However, after ChatGPT spread all over the world, OpenAI chose closed-source development, and the model codes such as GPT-3 and GPT-4 have become the trade secrets of OpenAI.

  The so-called open source is open source code.For example, once the big model is declared open source, anyone can publicly obtain the model source code, modify it or even redevelop it within the scope of copyright restrictions. To make a simple analogy,The source code is like the manuscript of a painting, and everyone can fill in the colors according to the manuscript to create their own artistic paintings.

  Closed source is just the opposite of open source.Only the source code owner (usually the software developer) has the power to modify the code, others can’t get the "manuscript" and can only buy the finished product from the software developer.

  The advantages and disadvantages of open source and closed source are very obvious. After open source, the big model will undoubtedly attract more developers, and the application of the big model will be more abundant, but the corresponding supervision and commercialization will become a difficult problem, which is prone to the embarrassing situation of "making wedding clothes for others".After all, open source considers ecological co-prosperity, and it is difficult to figure out the economic account of how much money can be earned at this stage, and these problems happen to be opportunities to close the source.

  Open source or closed source, this is a big model of life and death, the international giants have given the answer.

  Meta, the parent company of Facebook, released the big model Llama2 last month, which is open source and free for developers and business partners, while OpenAI firmly chose GPT-4 closed source development, which not only can maintain OpenAI’s leading position in the generative AI industry, but also can earn more revenue. According to the authoritative magazine Fast Company,OpenAI’s revenue in 2023 will reach 200 million US dollars, including providing API data interface services and subscription service fees for chat bots.

  Domestic big models have gradually begun to "go their separate ways".Alibaba Cloud’s General Meaning ModelAs early as April this year, it was announced to be open to enterprises, and the open source of Qwen-7B model will go further.ERNIE Bot of BaiduIt has also recently announced that it will gradually open the plug-in ecosystem to third-party developers to help developers build their own applications based on the Wenxin model.

  In contrast, Huawei does not take the usual path. When the Pangu Big Model 3.0 was released, Huawei Cloud publicly stated that,Pangu modelThe full stack technology is independently innovated by Huawei, and no open source technology is adopted. At the same time, Pangu Big Model will gather numerous industry big data (involving industry secrets, etc.), so Pangu Big Model will not be open source in the future.

  The big parameters are still small and beautiful.

  In addition, the open source of Qwen-7B model brings another thought:How many parameters do we need a big model?

  There is no denying that,The parameter scale of the large model is constantly expanding.Take the GPT model under OpenAI as an example. GPT-1 only contains 117 million parameters, and the parameters of GPT-3 have reached 175 billion, which has increased by more than 1000 times in a few years, while the parameters of GPT-4 have exceeded the trillion level.

  The same is true of large domestic models. Baidu Wenxin model has 260 billion parameters, Tencent mixed-element model has reached 100 billion parameters, Huawei Pangu model has been estimated to be close to GPT-3.5, and ali tong Yida model has officially announced 10 trillion parameters … …According to incomplete statistics, there are at least 79 large-scale models with over 1 billion parameters in China.

  Unfortunately, the larger the parameter, the stronger the capability of the large model. At the World Artificial Intelligence Conference, Wu Yunsheng, vice president of Tencent Cloud, has a very appropriate metaphor: "Just like athletes practicing physical strength, weightlifters need to lift 200 kilograms of barbells, and swimmers need to lift 100 kilograms. Different types of athletes don’t need everyone to practice 200 kilograms of barbells."

  As we all know,The higher the parameters of the large model, the more resources and costs are consumed.However, it is not necessary to blindly pursue "large scale" or "high parameters" to deepen the vertical large-scale model of the industry, but to formulate relevant model parameters according to customer needs. For example, the BioGPT-Large model has only 1.5 billion parameters, but its accuracy in biomedical professional tests is better than that of the general model with 100 billion parameters.

  Sam Altman, co-founder of OpenAI, also publicly stated that OpenAI is approaching the limit of LLM (Large Language Model) scale. The larger the scale, the better the model is, and the parameter scale is no longer an important indicator to measure the quality of the model.

  Wu Di, the head of intelligent algorithm in Volcano Engine, has a similar view. In the long run, reducing costs will become an important factor in the application of large models. "A well-tuned small and medium-sized model may perform as well as a general large model in a specific job, and the cost may be only one tenth of the original."

  At present, almost all domestic science and technology manufacturers have got tickets for big models, but the real road choice has just begun.

"Three Threads" in front of Xiaomi Automobile | Titanium Garage

Image source: vision china

Xiaomi car has not yet entered the market, and Lei Jun has begun to be anxious.

Recently, Lei Jun, chairman of Xiaomi Group, said in an interview with CCTV that he was worried that Xiaomi’s car would not catch fire as soon as it came up, and that everyone would buy it. "It was all kinds of anxious emotions."

In fact, since the day when Officer Lei announced the construction of the car, Xiaomi Automobile has always brought its own traffic attributes, but this is a "double-edged sword"-

  • On the one hand, Xiaomi automobile has gained great attention from the public without blowing off dust. Even though the government has not disclosed the progress of Xiaomi’s car-making, new news has been constantly revealed and even leaked many times.
  • On the other hand, the public’s high expectation of Xiaomi car is undoubtedly the pressure of Xiaomi to build a car. The public’s hung appetite needs to be fed to avoid traffic.

Although Xiaomi automobile has not entered the market, it has attracted much attention. In the face of huge traffic dividends, Xiaomi’s approach is not to increase exposure, but to maintain a "mystery."

It is reported that Xiaomi will launch a car-related event on December 28, but Xiaomi has not officially announced it. This kind of publicity rhythm is rare and abnormal in the industry.

But in any case, the debut of Xiaomi’s first model has entered the countdown, and the first shot of Xiaomi’s car will eventually be revealed. However, before Xiaomi Automobile officially entered the market, there were many difficulties ahead.

"Net red face" get together, can Xiaomi SU7 break through?

In November this year, Xiaomi Automobile landed in the new batch of new car declaration catalogue of the Ministry of Industry and Information Technology, and the first product of Xiaomi Automobile was also made public for the first time.

It can be seen that the appearance of Xiaomi’s first car is similar to that of Porsche Taycan, and it is similar to the models of Zhijie S7, Extreme Krypton 007, Extreme ROBO-02 and Star Road Interstellar ES.

Comparison between Xiaomi SU7 and competing models

According to the specific parameter information, the length, width and height of the first car "SU7" of Xiaomi are 4997mm/1963mm/1455mm, and the wheelbase is 3000mm, which is longer than the potential competitors such as Model 3, Tucki P7, Zhijie S7, and Extreme Krypton 007.

According to external sources, Xiaomi’s first car may be divided into three versions, namely, the regular version, the Pro version and the MAX version, and the price may be in the range of 190,000-300,000.

We should know that the B-class sedan market of 200,000-300,000 is the most competitive position at present. In this market segment, there are old faces such as BYD Han EV, Tesla Model 3 and Tucki P7, as well as the newly listed Zhijie S7, Yinhe E8 and Extreme Krypton 007.

Judging from the performance of the players on the field, BYD Han EV and Tesla Model 3 have gained a firm foothold, and sold 12,841 vehicles and 23,999 vehicles respectively in November, making them the two major peaks.

Among the new entrants, Zhijie S7 is an ecological product of HarmonyOS Zhixing, which uses Huawei’s latest technologies in chassis, safety, cockpit and intelligent driving, and is equipped with Huawei’s intelligent driving solution ADS 2.0;; Geely Group sits behind Krypton 007 and Galaxy E8.

In other words, Xiaomi SU7 is about to enter a battlefield surrounded by enemies.

According to the published application information, we can only know that Xiaomi SU7 will use lithium iron phosphate battery, and the supplier is Xiangyang Fudi Battery Co., Ltd. under BYD. The peak power of the driving motor is 220kW, and the supplier is United Automotive Electronics Co., Ltd.. In addition, the vehicle will provide optional items including side fender lettering, ETC, exterior rearview mirror, skylight glass, laser radar and so on.

Although more specific configuration information about Xiaomi car has not been made public, from the perspective of Xiaomi’s genes, its accumulation on mobile phone terminals makes it potentially competitive in smart cockpit.

In October this year, Xiaomi released "澎湃 OS" to open up the ecology of people and cars. Through the opening of the bottom layer of cross-scenario and cross-ecological software, Xiaomi Automobile got the card of "cross-terminal and cross-scenario experience", and the car companies in the industry currently have this kind of simultaneous layout of mobile phones and car terminals. There are three manufacturers: Geely, Weilai and Huawei.

Whether Xiaomi can release the advantages of drivers’ interconnection to form competitiveness after entering the automobile market is unknown, but the game between Xiaomi SU7 and Zhijie S7, Yinhe E8 and other products in drivers’ interconnection is actually that Xiaomi is once again facing Huawei, Meizu and other old rivals.

In addition to the competitiveness of product configuration, the most uncertain thing about Xiaomi Automobile at present is the pricing problem-the pricing is too high to gain market recognition and affect sales; If the price is too low, the standard line of brand impression will be set, and at the same time, factors such as profit rate need to be considered. At the same time, the intensified price war also makes the pricing of Xiaomi Automobile more difficult.

Faced with the layers of new and old players, Xiaomi SU7 has officially started the breakout campaign since it entered the arena.

How can the latecomers make up for the battle of opening the city with wisdom?

"The goal of Xiaomi Automobile is to enter the first camp of the autonomous driving industry in 2024", which is the swear words released by Lei Jun in his annual speech in August 2022.

In Lei Jun’s view, autonomous driving is the key to the success of smart electric vehicles. Xiaomi will also choose autonomous driving as the breakthrough direction of smart electric vehicles and unswervingly follow the path of full-stack self-research.

Before this path was made public, Lei Jun began to set up an autonomous driving R&D team with L4 as the goal as early as 2021, and integrated the artificial intelligence laboratory, Xiaoai team, mobile phone camera department and other departments of the group to provide cooperation for intelligent driving development.

According to the information revealed by Lei Jun in his speech, the first phase of Xiaomi Auto’s self-driving project invested 3.3 billion yuan in research and development. The team covers sensors, chips, sensing control algorithms, simulation technology, high-precision maps, high-precision positioning, tool chains, training capabilities and other automatic driving full-stack technical talents.

Later, external news pointed out that the person in charge of Xiaomi’s auto-driving was Ye Hangjun, a veteran of Xiaomi. He joined Xiaomi in 2012, and was first responsible for cloud technology, then for the artificial intelligence department, and then switched to the auto-driving industry.

According to Lei Jun’s interview with CCTV, Xiaomi invested 3,400 engineers in the first car and invested more than 10 billion in research and development.

Smart driving industrial chain enterprises invested by Xiaomi

In order to quickly make up lessons in autonomous driving, Xiaomi invested in a large number of upstream and downstream companies in the industrial chain before the layout of autonomous driving, including black sesame intelligence, an automobile chip manufacturer; Automatic driving manufacturers Momenta, Zhixing, Zongmu Technology, etc.; Laser radar manufacturer Hesai Technology, etc., and in 2021, it wholly acquired the autopilot company DeepMotion Tech Limited, which has a layout in the fields of high-precision positioning, high-precision maps and 3D scene reconstruction.

However, the demonstration of Xiaomi’s auto-driving technical ability still stayed in August, 2022. At that time, the video data showed that Xiaomi’s auto-driving system could realize the functions of calling vehicles with one button, automatically entering the ramp, actively changing lanes to overtake, making unprotected turn/U-turn, automatically bypassing temporarily stopped vehicles/accident vehicles/roundabout, and autonomous parking service. After the vehicle is parked and parked, it can also be automatically charged by the mechanical arm.

After more than a year of R&D iteration, it is still unknown whether these capabilities can be mass-produced. However, the domestic smart car market has ushered in an unprecedented development period, and the competition of car companies in intelligent driving has moved from high-speed NOA to urban NOA, and they are competing to be involved in the battle of opening the city NOA.

Some manufacturers have experienced the game of choosing high-precision maps, and the algorithm has also been iterated, reconstructed and revised, while Xiaomi Automobile has always been in a silent state.

Intelligent driving ability is on the one hand, and more importantly, engineering landing and verification. It needs a certain amount of training and testing on the open road. Even if Xiaomi’s first model has the ability of NOA in the city as soon as it comes on the stage, it still needs to catch up with the players who are relatively the first to lay out in the open speed.

Liu Yilin, Senior Director of Autopilot Products in Xpeng Motors, recently publicly talked about the importance of real vehicle data to the verification of urban assisted driving test. He mentioned that "it is risky to rely on shadow mode on a large scale in the cold start stage, because there is no real vehicle data, and whether the shadow mode itself is reliable or not has not been verified. Only by combining the real vehicle verification+simulation mileage+shadow mode, combined with the Bad Case of a large number of production vehicles, can the functional safety be more securely guaranteed. "

According to various plans, Huawei NCA plans to be available nationwide by the end of this year, Xpeng Motors NGP plans to expand to 50 cities by the end of this year, and Weilai NOP+ will open 60,000 kilometers by the end of this year. All car companies are accelerating the landing of urban driving assistance capabilities, and all of them have a certain amount of real vehicle data running on the road.

Xiaomi Automobile, which wants to enter the first camp of the autonomous driving industry in 2024, should compress the competition in product experience, online time and the number of Kaesong into a more compact time range, so as to be able to compete with other automobile companies. This is not only a technical competition, but also a sales competition and a resource competition.

Can the trial of productivity hell stand up?

Entering the manufacturing process is the beginning of Xiaomi Automobile’s mass production.

In an interview with CCTV, Lei Jun said, "Cars are really complicated. I am particularly worried that if they don’t catch fire, everyone won’t buy them; What is more worrying is that if everyone comes to buy it, it will take a year or two to wait, and it will definitely be miserable. "In fact, it refers to the management of supply chain and factory capacity.

Screenshot of the new car declaration catalogue of the Ministry of Industry and Information Technology

It can be seen from the information of the new car declaration catalogue of the Ministry of Industry and Information Technology that the qualification subject of Xiaomi Automobile is "Beijing Automobile Group Off-road Vehicle Co., Ltd." and the production address is "No.21 Courtyard of Huanjing Road, Beijing Economic and Technological Development Zone", that is, the location of Xiaomi Automobile’s self-built factory.

Although it is still doubtful whether Xiaomi’s automobile production qualification has fully landed (according to the Regulations of the National Development and Reform Commission on Automobile Industry Investment Management in 2018 and the Regulations of the Ministry of Industry and Information Technology on New Energy Automobile Production Enterprises and Product Access Management in 2020, the new car manufacturing company needs to obtain the two qualifications of the National Development and Reform Commission and the Ministry of Industry and Information Technology at the same time), it can be confirmed from the application information that Xiaomi will produce "Xiaomi SU7" in its own factory.

Xiaomi Automobile Factory has two projects, one of which covers an area of about 720,000 square meters and has an annual production capacity of 150,000 vehicles. At present, it has been completed and started to enter the trial production stage. The first product prototype has been off the assembly line and is ready for mass production. The second phase is scheduled to start in 2024 and be completed in 2025.

Lei Jun once said that the first model of Xiaomi Automobile plans to sell 100,000 vehicles in the first year and deliver 900,000 vehicles in the next three years. This means that from 2025 to 2027, Xiaomi Automobile needs to produce and sell at least 300,000 vehicles every year.

Although Xiaomi has experience in supply chain management and manufacturing in the field of mobile phones, it is still a novice in the automobile industry. The automobile industry, which is known as the "pearl in the crown of human manufacturing", not only has a long supply chain, but also has a difficult capacity management. Even the head car companies such as Tesla and BYD can’t escape the torture of "capacity hell".

This year’s "Biography of Musk" recorded the whole process of Tesla Model 3′ s experience of productivity hell. Even the Tesla factory with high automation degree will be stuck in the production line because of a slight mistake or a certain part. An unimpeded production line needs more time to run in and accumulate a lot of invisible knowledge in management and technology besides the initial capital investment and the design of the production line.

The difficulty of capacity management lies in supply chain management. A supply chain manufacturer once told Titanium Media App that Xiaomi’s early negotiations with the supply chain gave an expectation of "100,000 in the first year". Usually, suppliers will arrange production according to demand, but when the production capacity is tight, some suppliers will tilt resources based on strategic considerations, which will undoubtedly increase the difficulty of supply management of car companies.

This year, Tucki G6 Max had affected the delivery speed because the laser radar supplier was not prepared enough; In the delivery of Krypton 001, the production capacity of Krypton 001 was slow due to insufficient suspended supply.

Xiaomi Automobile Factory only entered the trial production stage in September this year, producing about 50 prototype cars every week. For Xiaomi automobile, which has not experienced actual production, supply chain management and capacity management are all thresholds that it has to cross.

There is no "newcomer protection period" in the automobile market.

Xiaomi once entered the mobile phone market as a subversive, but in the matter of building a car, Xiaomi is actually no different from many new force car makers. There may be no shortage of pits to step on and classes to make up.

Different from the new car companies that have already entered the market, Xiaomi needs to face the cruel competition in the automobile industry when it enters the market. It is a consensus in the industry that the new energy vehicle market enters the knockout stage. Whether it is a three-year boundary or a five-year competition period, the first shot is particularly important for Xiaomi Automobile, which arrived late.

After all, in the face of fierce market competition, there is no so-called newcomer protection period.

(This article is the first titanium media App, written by Xiao Man, edited by Sharla Cheung)

Registration time for CET-4 and CET-6 is in the first half of 2024.

Thirty-one regions across the country announced the registration time for the National College English Test Band 4 and Band 6 in the second half of 2023.

Heilongjiang test area: from 14: 00 on September 18th to 17: 00 on September 26th.

Changjiang professional college, Sichuan Test Area: 8: 00 on September 15th-17:00 on September 26th, 2023.

Guizhou Province: From 12: 00 on September 13th to 17: 00 on September 26th.

Inner Mongolia: 10: 00 on September 15th-17: 00 on September 25th.

Ningxia: From 14:00 on September 14th to 17:00 on September 26th.

Anhui: 10: 00 on September 19th-17:00 on September 26th.

Guangdong-south university of science and technology of china: 12: 00 on September 18th-17:00 on September 26th.

Jilin: 10: 00 on September 18th, 2023-17: 00 on September 26th.

Hebei test area: it starts at 6:00 am on September 19th and ends at 16:00 pm on September 26th.

Jiangxi: From 14: 00 on September 15th to 17:00 on September 22nd.

Hainan: September 13th to September 22nd.

Guangxi: From 14:00 on September 14th to 12:00 on September 20th.

Shanghai: From 14:00 on September 14th to 14:00 on September 26th.

Henan —— Henan Agricultural University/Henan Polytechnic University: 8: 30 on September 15th to 17:00 on September 22nd.

Hubei: From 11:00 on September 15th to 17:00 on September 25th.

Shanxi: The start time is 8:00 on September 18th, and the deadline is subject to the regulations of each test center.

Liaoning: from 10: 00 on September 14th to 17: 00 on September 26th.

Zhejiang: 6: 00 on September 14th-17:00 on September 26th.

Jiangsu: 12: 00 on September 14th-17:00 on September 23rd, 2023.

Fujian-Xiamen University: 9: 00 on September 13th-17:00 on September 23rd.

Hunan-Hunan College of Medicine: from 15:00 on September 15th to 16:00 on September 22nd.

Shaanxi-Xijing College: from 14: 00 on September 16th to 17: 00 on September 26th.

Gansu: 10: 00 on September 15th-17:00 on September 26th.

Qinghai: From 12: 00 on September 18th to 17: 00 on September 21st.

Xinjiang-Xinjiang Communications Vocational and Technical College: 10: 00 on September 12th-23:00 on September 17th.

Beijing: From 10: 00 on September 13th to 17:00 on September 26th.

Tianjin: From 14: 00 on September 18th to 17:00 on September 26th.

Yunnan-Haiyuan College of Kunming Medical University: 13: 00 on September 18th-12:00 on September 26th.

Chongqing-Chongqing Jiaotong University: 15: 00 on September 15th-17:00 on September 26th.

Shandong: 9: 00 on September 19th-17:00 on September 26th.

Tibet: 10: 00 on September 19th-18:00 on September 26th.