Lets take off the table the obvious answers of Barbarossa, declaring war by Germany and Italy on the U.S, and the first two waves at Pearl Harbor as choices.
Before I attempt to answer your question, I’d like to spend a little time exploring why Germany and Japan made the decisions they made in the first place. In 1940, Britain produced more military aircraft than Germany. In addition, large numbers of U.S.-built planes were sent to Britain for use against Germany, with plans to multiply that number several-fold over the coming years. Germany needed to vastly expand its own production to avoid destruction from the air. For that, it required more resources, a larger labor force, more oil, and more industrial capacity. Its leaders were also gravely concerned about the effect the British food blockade was having on Germany’s food supply. Hitler felt the answers to Germany’s problems could be found to the east, in the Soviet Union. The Russian Army fought poorly during WWI, and the Red Army embarrassed itself during its invasion of Finland in 1940. On the other hand, Hitler believed war between Germany and the U.S.S.R. was inevitable, giving him an incentive to strike quickly, before the Soviets were ready. In 1941, Germany killed or captured fifteen Soviet soldiers for every German soldier lost, resulting in the death or capture of over 4.1 million Soviet soldiers. By the battle of Kursk in 1943, the German combat effectiveness ratio would be reduced to 3:1; and at Stalingrad the Soviets achieved a nearly 1:1 ratio. The Soviets had vastly more people than Germany, so anything remotely approaching a 1:1 ratio spelled certain doom for Germany.
1942 was the pivotal year for the Axis. Either Germany would strike a decisive blow against the Soviet Union, or else it would doom itself to defeat. Both Hitler and Stalin understood this. Hitler’s plan had been to take the Caucasus. Had he done so, he would have controlled 90% of the Soviets’ oil supply. Also, the Caucasus contained farmland, needed mineral deposits, industrial capacity, and a respectably large population, all of which were needed by German wartime industry. (By 1942, captured Soviet soldiers were starving to death, despite Hitler’s orders that they be fed, because Germany lacked the food with which to feed them. As a result, Germany lost millions of people who–up until their starvation–had been put to work in German factories making weapons.)
Hitler’s decision to declare war on the U.S. was motivated by four beliefs. 1) The belief that over the short and medium term, the U.S.'s attention would be fully occupied by Japan. 2) the belief that 1942 was the key year of the war, for the reasons described above. The opportunity to sink the large numbers of American ships sending military aid to the Soviets was one Hitler was reluctant to pass up. 3) The belief that the U.S. had already converted its factories into an engine of war against Germany. 4. The belief that in the long run, FDR and other members of the pro-war faction would succeed in getting the U.S. to go to war; just as the American pro-war faction had ultimately succeeded in causing the U.S. to enter WWI.
With the benefit of 20/20 hindsight, it’s clear where both Hitler and the Japanese leaders went wrong. Both were guilty of vastly overestimating Japan’s ability to wage war against the U.S. Also, Japan’s leaders totally misjudged the American public’s reaction to their Pearl Harbor attack. The U.S.'s kill Germany first strategy began creating problems on the ground for Germany in late '42 (Algeria). By '43 the U.S. Army, together with the British, had pushed into Italy. Hitler had given Germany too short a window to exert pressure on the Soviets before the German Army would be diverted by the threat of the American Army in the west.
You said that the obvious answers to your question (such as Barbarossa and the declarations of war on the U.S.) were off the table; and that you were searching for some other “worst mistake” the Axis had made. For the purposes of your question, I will assume that Germany’s decision to attack the Soviet Union in June of '41 is locked in; just as the German and Japanese decisions to go to war against the U.S. in December of '41 were also locked in. The question then becomes, is there anything Germany or Japan could have done to exit one or more of their major wars after having entered them?
Neither Germany nor Japan could launch a credible invasion of North America. Germany had a very small window of opportunity during which it might have invaded Britain. However, it lacked the surface fleet and transport capacity necessary to take advantage of that opportunity.
During Hitler’s administration, the German Navy consistently received around 10% of overall military funding. It’s easy to imagine German naval spending being double what it actually was in the period from '37 - '40. The objective would have been to build up a credible surface fleet and transport capacity. After the fall of France, Germany would have used this fleet in a desperation move to take Britain as quickly as possible. This invasion would not necessarily have succeeded. But it might have–especially if, after the fall of France, Germany fully committed itself to making the invasion work. (And was willing to launch the invasion with a less than 100% chance of success.) A successful invasion would have cleared off Germany’s western front, allowing it to fully focus on the coming invasion of the Soviet Union. Germany’s failure to prepare for the invasion of Britain is one possible candidate for worst mistake.
Assuming Germany had been unable to launch Operation Sea Lion, it still might have been able to win its war against the Soviet Union. To do so, Hitler would have needed to put the right people in charge. Ideally, this would have meant putting Speer in charge of armaments production in 1939 at the latest; with von Manstein given overall responsibility for preparing for war in the east. Von Manstein and Speer would have worked closely with each other to determine the weapons and equipment needed by the German military; and Speer would have done whatever was necessary to provide. Von Manstein would also have been given permission to fight the war as he wanted. Von Manstein’s approach would have been very opportunistic and free-flowing; with the intended goal of inflicting maximum harm on the enemy while minimizing the harm to the German Army. He might sometimes have retreated, in order to tempt the Soviets into an ill-advised advance. Then, when the Soviets were in the process of moving forward–away from their fixed defenses–von Mannstein would counterattack and destroy the Soviet force in question. (The series of battles he directed in the wake of the Stalingrad defeat were a good example of this.) The overall effect of von Manstein’s generalship would have been to maximize Soviet losses while minimizing German losses. He would also have conquered large quantities of ground.
Most of the German generals opposed the planned blitzkrieg of France, and supported the idea of the attack at Kursk. (The latter was an attack the Soviets obviously knew was coming, and against which they had prepared.) Von Manstein opposed the idea of attacking Kursk, and suggested the Germans attack in the south instead–where the Soviets were not expecting. In addition, he worked together with Guderian to develop the brilliant plan to blitz France. (He opposed the German generals’ idea of re-using the WWI-era Schlieffen Plan.) He was an outside-the-box, unconventional thinker, and rose above the generalship of the German generals or of Hitler. This was the man who should have been given overall responsibility for the war in the east!
As long as I’m on the “putting the right people in charge” theme, I should also address the subject of Goering. In 1940, Goering greatly reduced the number of engineers allocated to the development of jet aircraft. As though determined to make 1940 a banner year for himself, he persuaded Hitler that the British Expeditionary Force could be destroyed from the air. (And in so persuading, he paved the way for the Dunkirk evacuation.) Determined to top these achievements in 1942, he promised Hitler that the Stalingrad pocket could be adequately supplied by air. Taking the city had been very difficult in the first place–the Germans experienced a 1:1 exchange ratio–so abandoning it would have been very painful. Goering’s ability to over-promise and under-deliver directly led to the German Army’s worst defeat of the war. Practically anyone would have been preferable to Goering, but I’d like to nominate Kesselring for leading the Luftwaffe.
On the production side, putting the right people in charge would have caused Germany to have more and better aircraft, tanks, and other weapons. The increased number and quality of aircraft could have been decisive in the Battle of Britain. Cooperation between Speer and von Mannstein could have led Germany to produce large numbers of effective yet mechanically simple tanks for use against the Soviet Union. Putting a man of some intelligence in charge of the Luftwaffe could have led to Germany getting jet aircraft significantly sooner than it did. As it was, Germany began using small numbers of jet aircraft in combat in '44. It wasn’t just jet aircraft that Goering had slowed. In general, he opposed Hitler’s faith in “miracle weapons,” thereby causing him to lie about the performance of some of those so-called miracle weapons. For instance, he claimed that a radio-guided bomb Germany had used against a British ship hadn’t worked, that the British had jammed the radio signals, and that the ship had to be sunk by conventional means instead. This kind of misinformation prevented Germany from intelligently allocating resources to the best available miracle weapons projects, while refraining from funding dead ends.
Had the right people been put in charge back in '38, it’s quite possible that the German summer offensive of ‘42 would have succeeded in its objective. (The objective was to take the Caucasus.) If that objective seems overly ambitious in hindsight, it’s largely because the German military was significantly weaker and less effective than it would have been, had the right people been put in charge. In 1942, the Soviets outproduced Germany by a margin of 3:1 - 4:1 in most major land weapons categories, and by 2:1 with respect to military aircraft. Had Speer been given a few years to build up Germany’s war machine, I do not think he would have let that happen. Hitler’s failure to put the right people in charge is another candidate for “worst Axis mistake.” (This is not to suggest the Allies were perfect about putting the right people in charge–they weren’t–but the Allies’ advantage in industrial capacity and manpower meant they could afford far more mistakes than Germany.)
The subject of chemical weapons is an interesting one. Germany’s chemical weapons research was significantly more advanced than the Allies’. However, it was understood that, if one side used chemical weapons, the other side would immediately and massively retaliate in kind. Hitler’s decision to refrain from using chemical weapons was made with that understanding in mind. However, chemical weapons programs were secret, and the Germans didn’t realize how far ahead they were in chemical weapons research.
Suppose Germany had begun using advanced forms of nerve gas in its war against the Soviets. You didn’t actually have to breathe this stuff in for it to kill you: mere exposure to the skin was enough. Had this gas been stored in artillery shells and lobbed against Soviet targets, the results on the battlefield would have been devastating. This weapon would have been especially effective against large numbers of Soviet soldiers concentrated in relatively small areas–such as the area in front of Moscow. German soldiers would have needed gas masks and goggles to deal with retaliatory Allied chemical weapons attacks. Soviet soldiers, on the other hand, would have required complete head-to-toe skin covering, with some material which would allow zero air penetration. It would have taken quite some time to procure such apparel for Soviet soldiers. In the meantime, Germany could have drastically thinned out the Soviets’ numerical advantage, while conquering large amounts of land in the process. (In contrast, Germany would have prepared its soldiers for all this in advance, by giving them the gear necessary to deal with Soviet chemical weapons attacks.)
There is the fear that, if the Germans used chemical weapons on the battlefield, the Allies might have retaliated by using them against German civilian populations. The use of chemical weapons would have necessitated a significant commitment to Germany’s rocketry program; including the longer-ranged designs Werner von Braun had planned. Had an Allied nation launched a chemical attack on a German city, Germany would have responded by using rockets to deliver a chemical payload to one of the offending Allied nation’s cities.
There are three different things Germany could have done to have potentially knocked a major Allied participant out of the war. (Preparing for the invasion of Britain, putting the right people in charge, and using chemical weapons against the Red Army.) Of the three, I’m most inclined to vote for the middle one as the biggest mistake. Once you have the right people in charge, they will tend to make whichever decisions need to be made to get the job done.