6 Days to Fight Hackers
The work of Internet companies is difficult to avoid not dealing with hackers. Almost every day every minute, hackers scan on the company’s website.
Some are looking for gaps injected by SQL, and some are looking for possible vulnerabilities in online servers. Most are scanned by various hacking detection tools. Of course, more advanced intrusions require a lot of manual assistance.
Under normal circumstances, the website is hacked because the developers usually do not have security awareness. Severe hackers are often very familiar with the platform’s business and internal processes. Many vulnerabilities are logically analyzed.
Of course, all hacking attacks are purposeful, and 99% is because of the huge profits hidden in it. For instance, between 2015–2017, many Internet finance companies suffered from hackers extortion, and in the end, they all made money.
I have been reading on how security professionals fight online with hackers for many years and have accumulated some cases. As a result, I will share with you what happened in a fictional company last year.
An event caused by a slow customer feedback operation
Day 1
On a certain day in 2018, the customer service received feedback from individual users. A certain interface of the company’s app was slow to respond during operation. The customer service reported this problem to the technical staff of the company.
After receiving this feedback, the technical staff repeatedly tested it in the production environment and did not reproduce the problem, so the customer feedback may be a network problem, you can try another network.
Day 2
The customer service received more user feedback, and the mobile phone is very slow when operating a certain function. The maximum time is to wait for more than 1 minute. Customer service feeds back relevant information to the technology and raises the problem level by level.
On the technical side, the problem is raised to the technical manager, and the cause of the problem is then investigated. The technical manager found that this slow phenomenon will recur once after every 10 tests.
It seems to be another occasional problem. The occasional bug is the most difficult to deal with. The technical manager took the R&D staff to adjust the front-end business logs on the server. After analysis, no specific reason was found.
Day 3
The company has a core agent to report this problem to the technical CEO, and the CEO arranges the technical director to pay attention to this problem. The technical director asked to log all the key nodes of the access path to see which part of the operation is slow.
After the technical manager’s investigation, it is occasionally slow when the app operates a certain function, but this operation also has a cache as a support, and the cache utilization rate is not very high.
Since the problem is infrequent and the subsequent cache utilization rate is not high, then it should not be a bug in the program, it should be a problem in other areas, and it is requested to investigate whether it is a server problem.
Let’s illustrate for everyone to help you understand the technical background.
When the front-end user uses the mobile phone to operate a certain function, the request is distributed to the front-end Nginx through the firewall router, and the Nginx is distributed to the next five business servers through a balanced load.
A set of data needs to be invoked during business operations, and these data will be cached in the back-end cache server cluster according to business rules to relieve the pressure on the back-end database.
Day 4
After cooperating with the operation and maintenance personnel, it was found that among the five business servers distributed, one of them had relatively high pressure. Any request for distribution to this server would be slow.
Because the online server has been used for many years, it is suspected that it is a problem caused by the ageing of the server configuration, and then the more stressful server is offline and seamlessly switched to the new server.
After half an hour, continue to observe, and found that another server also has a problem of heavy load pressure. A lot of solutions have been changed later, such as reducing to 3 servers to support or increasing to 9 servers to support.
When switching from half an hour to two hours, the pressure of 1–2 servers will suddenly come up and keep a relatively high amount of access. After these tests, it is ruled out that the server is the problem.
Immediately after excluding whether the Nginx distribution strategy has a problem, resulting in too many requests distributed by a server, after trying to change the Nginx distribution strategy several times, the problem will still be reproduced and the Nginx distribution exception will be ruled out.
Day 5
The engineers continued to track down, and the server thread with higher load pressure came down and compared with the normal server, and found that the number of calls of a certain thread on the server with higher load pressure was abnormally high.
Continue to track this thread. It turns out that a function of forgetting the password is frequently called. Forgetting the password should be a common interface. Why is it called so frequently?
Then continue to track which IPs are calling these interfaces frequently. If you do not track, you will be shocked. The top 100 IPs that are most frequently called are listed and checked. It is found that they are all foreign IPs.
What Paris, Kampala, London, almost all foreign IP have, but as a domestic third-party payment company, how can foreign IP appear?
Everyone knows that when you forget your password on your mobile phone, most platforms use a mobile phone number as one of the conditions. If you do not register on this platform, you will report that the mobile phone number does not exist.
This is a very common function, but for hackers, this function can be extremely profitable.
Many competitors in the same industry need to tap other company customers, but how do you know who these platform customers are?
Directly through this interface, brute force cracking is enough.
Because the business handled by the company is domestic, as long as the script is directly written in Nginx, all IPs from abroad are shielded, which can temporarily solve the problem of attacks by foreign IPs.
The consequences of going online have ceased, the server is no longer overloaded, and app access is normal.
Day 6
Perhaps the hacker also reacted after night and began to use the domestic server proxy IP to call this interface with high frequency. Since the hacker used the domestic IP address, it must not block the domestic IP.
However, there is a front-end server after Nginx, and a very simple filter is added to the front-end server: setting an IP address for a certain period (configurable) can only access this interface 5 times, which can solve this problem.
Then limit the user as the dimension. A user can only access this interface 5 times within a certain period (configurable).
After the above methods are completed, everything is quiet, and there has never been a problem of service overload or slow response of user feedback operation.
But if the hackers have enough domestic IP, to prevent such problems, we will then upgrade the product to prevent this possibility of hackers.
We all know that there are many app pages now. To facilitate user operation, there is no graphical verification code when the password is forgotten, or it is relatively simple and can be cracked by the machine.
Therefore, a new generation of smart verification measures needs to be upgraded. Now many startups can provide this service, which prevents hackers from obtaining customer information through violent means from all dimensions.
To sum up
As far as I know, many Internet companies still have this kind of vulnerability. It is easy for hackers to get the user’s registered mobile phone number through brute force cracking. As long as they get the user’s mobile phone number, they can use various marketing strategies.
This is a competitive method adopted by many companies at present.
Sometimes, the product’s ease of use and security conflict. When designing a product, you should consider the related design of risk control with the technology, otherwise, problems will easily occur in the later stages of product operation.
There are many risk points in a large Internet company. Server risk points and business risk points need to be considered in advance. At the same time, the company must be equipped with corresponding security personnel to conduct routine security inspections on the company’s website.
Monitor the frequency of all access interfaces of the company, and periodically analyze all the IP access in the company as a chart to analyze which are normal requests and which are abnormal access.
Regularly check all requests for statistical analysis access, you can find many problems in advance.
The struggle between developers and hackers is a long-term job. Any company with a large number of visits will encounter similar problems. Do n’t panic if you encounter problems. Carefully check every detail, and you will find the answer to the problem.
