By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
Gulf PressGulf Press
  • Home
  • Gulf News
    • Saudi Arabia
    • UAE24/7
    • Kuwait
    • Qatar
    • Bahrain
    • Oman
  • World
  • Business
    • Market DataLive
    • Finance
    • Economy
    • Energy
    • Crypto
    • ForexHot
    • Tech
  • Sports
  • Lifestyle
  • Videos
Search
Countries
  • Saudi Arabia
  • UAE
  • Kuwait
  • Qatar
  • Bahrain
  • Oman
More Topics
  • Technology
  • Health
  • Entertainment
  • Crypto
  • Forex
  • Stocks
Site Links
  • Business Hub
  • Trending
  • Weather
  • Customize Interests
  • Bookmarks
  • Newsletter
  • Terms
  • Press Release
  • Advertise
  • Contact
© 2023 Gulf Press. All Rights Reserved.
Reading: AI experts are prepared to challenge powerful technology in “Humanity’s Last Exam”
Share
Notification Show More
Recent Saved
Workers in Bucharest face challenges as temperatures rise
World
Dress code guidelines for the Qatari government sector during office hours
Qatar
Court rules in favor of worker after company dismisses him for salary deductions over 6 years, awarding BD 27,000.
Bahrain
UAE to See Almost 30,000 New Millionaires in 5 Years
UAE
Proposed New Labor Law in Bahrain Targets Increasing Job Opportunities
Bahrain
Latest News
Ferrero’s Social Responsibility Project Kinder Joy of moving Beats Traditional PE Curriculum, Tapping into the Cognitive Functions, Motor Coordination and Life Skills of Students
Lifestyle
UAE Ranks Among Top Rugby Markets on TOD as British & Irish Lions Tour Kicks Off
Sports
Darven: A New Leap in AI-Powered Legal Technology Launching from the UAE to the World
Tech
Beat the Heat This Summer with a Chill Out on Dubai Marina
World
Historic Italian City of Assisi to Host International Exhibition “Jordan: Dawn of Christianity”
World
Jordan to Host Iraq in the Final Round of the Asian World Cup Qualifiers After Securing Historic Spot
Sports
The Myriad Redefines Student Living for a Mobile, Urban Generation
UAE
L’Oréal Paris Welcomes Yasmine Sabri as New Brand Ambassador for the Middle East
Lifestyle
Roland Garros 2025: A New Era of Viewing, A Tribute to Legends, and Moments to Remember
Sports World
Bitget Celebrates Bitcoin Pizza Day by Distributing Over 5000 Pizzas in Over 20 Cities Worldwide
Crypto
Bitget Protection Fund Maintains Strength with $561 Million Average Value in April 2025
Business Crypto
Moscow Metro Marks 90 Years with Technological Advancements and Heritage Events
Lifestyle World
Senkron Digital Showcases CyberPact Service at GISEC Global 2025
Gulf UAE
Muhammad Umair Saeed: The Billionaire Tech Architect Powering AI, Drones, Blockchain & Cybersecurity from Dubai to the World
Business Gulf UAE
Barcelona Aims to Recover from European Heartbreak as They Face Real Madrid in La Liga, Chasing Their Third Title of the Season
Sports
UAE with 47% and Qatar with 36% Surge in Viewing Minutes as TOD Unveils its Enhanced Streaming Experience TOD 2.0
Lifestyle
L’Oréal Paris Celebrates the Groundbreaking Launch of MELASYL at the Iconic Museum of the Future
Lifestyle
TOD Partners with Skyworth’s QVWi to Launch 4K Streaming Set-Top Box in MENA
Gulf Sports
Double victory at Red Dot: Eczacıbaşı Tiles Group’s VitrA and Villeroy & Boch brands recognised for their design excellence
Lifestyle
Your Next Destination in Dubai: Mövenpick Hotel Apartments Downtown Dubai – The Perfect Choice for Your Stay
Gulf UAE
Aa
Gulf PressGulf Press
Aa
  • Gulf News
  • World
  • Business
  • Entertainment
  • Lifestyle
  • Sports
  • Videos
Search
  • Home
    • Videos
    • Business Hub
    • Trending
  • Gulf
    • Saudi Arabia
    • UAE
    • Kuwait
    • Qatar
    • Bahrain
    • Oman
  • Business
    • Market Data
    • Crypto
    • Economy
    • Energy
    • Finance
    • Forex
    • Tech
  • More News
    • World
    • Lifestyle
    • Entertainment
    • Sports
Have an existing account? Sign In
Follow US
  • Terms
  • Press Release
  • Advertise
  • Contact
© 2023 Gulf Press. All Rights Reserved.
Gulf Press > Business > AI experts are prepared to challenge powerful technology in “Humanity’s Last Exam”
Business

AI experts are prepared to challenge powerful technology in “Humanity’s Last Exam”

News Room
Last updated: 2024/09/17 at 5:27 AM
News Room
Share
4 Min Read
SHARE

Seeking to challenge artificial intelligence systems that have been handling benchmark tests with ease, a team of technology experts has issued a global call for the toughest questions to determine when expert-level AI has truly arrived. This project, called “Humanity’s Last Exam,” is a collaboration between the Centre for AI Safety and startup Scale AI. The goal is to remain relevant as AI capabilities continue to advance in the future.

The call for tough questions comes after the unveiling of OpenAI o1, a new model developed by the maker of ChatGPT, which has excelled in reasoning benchmarks. Dan Hendrycks, executive director of CAIS, co-authored two papers in 2021 proposing tests for AI systems on topics like US history and competition-level math. AI systems were previously struggling to answer questions on these tests, but now they are excelling, rendering common benchmarks less meaningful.

Although AI has shown improvements in some areas, lesser-used tests involving plan formulation and visual pattern-recognition puzzles have revealed poor performance. As some researchers argue that planning and abstract reasoning are better measures of intelligence, Hendrycks emphasizes the need for abstract reasoning in tests like “Humanity’s Last Exam.” Additionally, privacy measures will be taken to ensure that AI systems’ answers are not based on memorization of common benchmarks.

The upcoming exam will consist of at least 1,000 crowd-sourced questions that are challenging for non-experts to answer. These questions are due on November 1 and will undergo peer review, with winning submissions receiving co-authorship and up to $5,000 in prizes sponsored by Scale AI. By providing harder tests for expert-level models, the organizers aim to measure the rapid progress of AI accurately. However, questions related to weapons will be restricted due to safety concerns.

It is evident that AI systems have made significant progress in certain benchmark tests, prompting the need for more challenging assessments to gauge expert-level capabilities. The “Humanity’s Last Exam” project aims to push AI systems to their limits to determine when they have achieved expert-level performance. With the involvement of the Centre for AI Safety and Scale AI, this initiative seeks to offer relevant and rigorous evaluations for AI systems as they continue to evolve.

In light of recent advancements in AI capabilities, traditional benchmarks may no longer serve as accurate measures of intelligence. Tests involving various topics like US history and competition-level math have shown significant improvements from AI systems, highlighting the need for more challenging assessments. “Humanity’s Last Exam” promises to provide a platform for rigorous testing, including questions that are difficult even for non-experts, ensuring that AI systems are truly pushed to their limits.

As AI researchers explore the nuances of intelligence measurement through various tests, the importance of abstract reasoning and planning skills in AI systems has been emphasized. The visual aspect of certain tests may not always be aligned with language models, leading to the need for diverse evaluation methods. By offering a comprehensive exam that prioritizes abstract reasoning, “Humanity’s Last Exam” aims to provide a holistic assessment of AI systems’ capabilities, setting a new standard for expert-level performance evaluation.

Sign Up For Daily Newsletter

Be keep up! Get the latest breaking news delivered straight to your inbox.
I have read and agree to the terms & conditions
By signing up, you agree to our Terms of Use and acknowledge the data practices in our Privacy Policy. You may unsubscribe at any time.
News Room September 17, 2024
Share this Article
Facebook Twitter Copy Link Print
Previous Article Sharjah Safari announces opening date and explains ticket prices and timings
Next Article Sean ‘Diddy’ Combs arrested by US federal agents over assault allegations – News
Leave a comment Leave a comment

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Stay Connected

235.3k Followers Like
69.1k Followers Follow
56.4k Followers Follow
136k Subscribers Subscribe
- Advertisement -
Ad imageAd image

Latest News

Ferrero’s Social Responsibility Project Kinder Joy of moving Beats Traditional PE Curriculum, Tapping into the Cognitive Functions, Motor Coordination and Life Skills of Students
Lifestyle July 14, 2025
UAE Ranks Among Top Rugby Markets on TOD as British & Irish Lions Tour Kicks Off
Sports July 6, 2025
Darven: A New Leap in AI-Powered Legal Technology Launching from the UAE to the World
Tech July 1, 2025
Beat the Heat This Summer with a Chill Out on Dubai Marina
World July 1, 2025

You Might also Like

Tech

Darven: A New Leap in AI-Powered Legal Technology Launching from the UAE to the World

July 1, 2025
Crypto

Bitget Celebrates Bitcoin Pizza Day by Distributing Over 5000 Pizzas in Over 20 Cities Worldwide

May 22, 2025
BusinessCrypto

Bitget Protection Fund Maintains Strength with $561 Million Average Value in April 2025

May 20, 2025
BusinessGulfUAE

Muhammad Umair Saeed: The Billionaire Tech Architect Powering AI, Drones, Blockchain & Cybersecurity from Dubai to the World

May 10, 2025
BusinessCrypto

RWA project FEXSE tokenises $600,000 Jacob & Co. luxury watch on the blockchain

April 14, 2025
Business

NTT DATA Business Solutions Expands Presence in UAE with Stronger Regional Leadership

March 28, 2025

Sustainable Moving Services: How Dubai’s Moving Companies Are Going Green

February 20, 2025
Tech

CNTXT and Oracle Strengthen AI Collaboration to Drive Innovation

February 14, 2025
//

Gulf Press is your one-stop website for the latest news and updates about Arabian Gulf and the world, follow us now to get the news that matters to you.

Quick Link

  • Privacy Policy
  • Terms of ue
  • Advertise
  • Contact

How Topics

  • Gulf News
  • International
  • Business
  • Lifestyle

Sign Up for Our Newsletter

Subscribe to our newsletter to get our latest news instantly!

I have read and agree to the terms & conditions
Gulf PressGulf Press
Follow US

© 2023 Gulf Press. All Rights Reserved.

Join Us!

Subscribe to our newsletter and never miss our latest news, podcasts etc..

I have read and agree to the terms & conditions
Zero spam, Unsubscribe at any time.

Removed from reading list

Undo
Welcome Back!

Sign in to your account

Lost your password?