AI Powered Document Processing with Google Document AI and Gemini

Learning About Document AI and Gemini AI

Lately, I've been learning more about using AI for document processing. I came across Google Document AI and Gemini AI, and they solve two different parts of the problem. Document AI is mainly used to read and extract information from documents. It can process PDFs, extract text, and understand the structure of the document. Then there's Gemini, which is better suited for understanding and reasoning about that extracted information.

PDF
 ↓
Document AI
 ↓
Extracted text
 ↓
Gemini
 ↓
Understanding
 ↓
Result

Document AI

It can process documents such as PDFs and extract the text and structure from them. For example, a CV could be converted into something like:

WORK EXPERIENCE

Software Engineer
ABC Company
Jan 2020 - Jun 2022

Senior Software Engineer
XYZ Company
Jul 2022 - Present

EDUCATION

Bachelor of Computer Science
...

Now I have the actual content of the document that I can work with. For larger documents, the file can also be stored in Google Cloud Storage and processed asynchronously using Document AI's batch processing.

Gemini AI

Once the document has been extracted, this is where Gemini becomes useful. Instead of asking Gemini to figure out everything from the original PDF, I can give it the extracted content and ask it to understand and analyze it.

For example:

Does this candidate have at least 5 years of professional experience?

Or:

Does this document contain a signature?

Or:

Does the document contain a Work Experience section?

Gemini can then extract the relevant information and return a structured response that my application can use.

Putting Them Together

For example, I can give Gemini a prompt like this:

You are a CV reviewer.

Requirement:
"The candidate must have at least 5 years of professional work experience."

Review the CV and identify all professional work experiences.

For each experience, extract:
- Company name
- Job title
- Start date
- End date
- Whether the candidate is currently working there

Then determine whether the candidate meets the 5-year requirement.

Return the result as JSON.

The response can then be structured like:

{
  "meets_requirement": true,
  "total_experience_years": 6.5,
  "experiences": [
    {
      "company": "ABC Company",
      "job_title": "Software Engineer",
      "start_date": "2020-01",
      "end_date": "2022-06"
    },
    {
      "company": "XYZ Company",
      "job_title": "Senior Software Engineer",
      "start_date": "2022-07",
      "end_date": "Present"
    }
  ]
}

The part that surprised me the most is that I don't need to train some fancy model myself. A lot of the work is simply figuring out what I want the AI to do and how to explain it properly.