Add API V2 from DMPonline

[gjacob24]

PR for issue #3586

This issue branch uses the doorkeeper gem to implement the Authorization Code Grant Flow (ACGF) part of the v2 API.
(The Client Credentials Flow (CCF) part of the v2 API (which is what the org-admins will use) will be implemented separately in a future piece of work.)

[github-actions]

</tr>

	1 Error
🚫	Please include a CHANGELOG entry. You can find it at [CHANGELOG.md](https://github.com/DMPRoadmap/roadmap/blob/main/CHANGELOG.md).

	1 Warning
⚠️	This PR is too big! Consider breaking it down into smaller PRs.

Generated by 🚫 Danger

[aaronskiba]

Thank you for this PR.

During today's DMPRoadmap Monthly all-team meeting I mentioned how there are also related DMPTool v2 API test files. Would you like me to add them to this PR?

[momo3404]

I'm seeing a lot of rubocop errors in here. This might be a difference in the configuration of Rubocop between the Roadmap repo and DMP Online. Let me know how you'd want to approach this. If possible, I could start a PR attempting to fix these.

[martaribeiro]

Here is the document with the API calls.
API_v2.pdf

Please let us know if needs changing

[gjacob24]

I'm seeing a lot of rubocop errors in here. This might be a difference in the configuration of Rubocop between the Roadmap repo and DMP Online. Let me know how you'd want to approach this. If possible, I could start a PR attempting to fix these.

@momo3404 apologies we missed this. I can do this and push it back next week, if that's ok.

[martaribeiro]

These two docs are more for developers support. It might be useful
[Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827
doorkeeper_gem_for_OAuth_2.0_flows.pdf
178/Authorization_Code_Grant_Flow.pdf)

[aaronskiba]

These two docs are more for developers support. It might be useful [Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827 doorkeeper_gem_for_OAuth_2.0_flows.pdf 178/Authorization_Code_Grant_Flow.pdf)

Awesome, thank you. Sorry, the links got a little bungled up in the comment. :P
https://github.com/user-attachments/files/23827180/doorkeeper_gem_for_OAuth_2.0_flows.pdf works.
Could you just provide the Authorization_Code_Grant_Flow.pdf link again?

[gjacob24]

These two docs are more for developers support. It might be useful [Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827 doorkeeper_gem_for_OAuth_2.0_flows.pdf 178/Authorization_Code_Grant_Flow.pdf)

Awesome, thank you. Sorry, the links got a little bungled up in the comment. :P https://github.com/user-attachments/files/23827180/doorkeeper_gem_for_OAuth_2.0_flows.pdf works. Could you just provide the Authorization_Code_Grant_Flow.pdf link again?

@aaronskiba Here you go:
Authorization_Code_Grant_Flow.pdf

Let us know if this doesn't work.

[aaronskiba]

I'm encountering the following when I test the GET /api/v2/heartbeat endpoint (I can also see that the endpoint is returning a 500 for DMPOnline):

$ curl http://127.0.0.1:3000/api/v2/heartbeat
NoMethodError at /api/v2/heartbeat
==================================

undefined method `name' for nil:NilClass

> To access an interactive console with this error, point your browser to: /__better_errors


app/views/api/v2/_standard_response.json.jbuilder, line 18
----------------------------------------------------------

``` ruby
   13   json.ignore_nil!
   14   
   15   json.server @server
   16   json.source "#{request.method} #{request.path}"
   17   json.time Time.now.to_formatted_s(:iso8601)
>  18   json.client @client.name
   19   json.code response.status
   20   json.message Rack::Utils::HTTP_STATUS_CODES[response.status]
   21   
   22   if response.status == 200
   23   

App backtrace

• app/views/api/v2/_standard_response.json.jbuilder:18:in `_app_views_api_v___standard_response_json_jbuilder__1455696549825964062_81440'
• app/views/api/v2/error.json.jbuilder:3:in `_app_views_api_v__error_json_jbuilder__3591388735008983784_81420'
• app/controllers/api/v2/base_api_controller.rb:69:in `handle_internal_server_error'
• app/controllers/api/v2/base_api_controller.rb:58:in `handle_exception'

Full backtrace
...

[aaronskiba]

Since we will be implementing the v2 API Client Credentials Flow in a future piece of work, maybe we should remove null: false on redirect_uri, as the code comment advises.

[aaronskiba]

I can see @payload[:message] being assigned within app/controllers/api/v2/base_api_controller.rb. But is @payload[:details] being set anywhere? Maybe it can be removed.

[aaronskiba]

I'm a little worried about directly inheriting from ApplicationRecord and simply mapping to the Doorkeeper tables in app/models/oauth_access_grant.rb, app/models/oauth_access_token.rb, and app/models/oauth_application.rb. These classes will not include Doorkeeper’s behaviour (validations, associations, etc.).

Currently, @client = OauthApplication.find(doorkeeper_token.application_id) if doorkeeper_token appears to be the only code that references these models (see app/controllers/api/v2/base_api_controller.rb), which should be safe. But I worry about issues arising in the future.

Unless this was implemented for a specific reason, I might suggest one of the following:

1. Remove these classes altogether

• These tables/models can instead be referenced via Doorkeeper::Application, Doorkeeper::AccessToken;, and Doorkeeper::AccessGrant. (e.g. @client = Doorkeeper::Application.find(doorkeeper_token.application_id) if doorkeeper_token)

2. Update each of the classes to inherit from the appropriate Doorkeeper::Application, Doorkeeper::AccessToken;, or Doorkeeper::AccessGrant.

class OauthApplication   < Doorkeeper::Application; end
class OauthAccessToken   < Doorkeeper::AccessToken; end
class OauthAccessGrant   < Doorkeeper::AccessGrant; end

[aaronskiba]

It might be even better to reuse the Plan.active(@resource_owner) scope for this query. Note that a bit of refactoring could be performed in that scope to make this more efficient (i.e. don't use includes(:template, :roles))

[momo3404]

Please let me know if I've done something wrong here, but I keep getting 302s when executing the GET command from the Authorization Code Grant Flow.
curl -i -X GET "http://127.0.0.1:3000/oauth/authorize?response_type=code&client_id=[my_client_id]&redirect_uri=[redirect_uri]" HTTP/1.1 302 Found x-frame-options: SAMEORIGIN x-xss-protection: 0 x-content-type-options: nosniff x-permitted-cross-domain-policies: none referrer-policy: strict-origin-when-cross-origin location: http://127.0.0.1:3000/users/sign_in content-type: text/html; charset=utf-8 cache-control: no-cache

But the command still worked, because after calling Doorkeeper::AccessGrant.first I found my authorization which I used for the POST command, and that worked perfectly. So is this a known error, or something I'm missing? I was also signed in when making the request.

[gjacob24]

I'm encountering the following when I test the GET /api/v2/heartbeat endpoint (I can also see that the endpoint is returning a 500 for DMPOnline):

$ curl http://127.0.0.1:3000/api/v2/heartbeat
NoMethodError at /api/v2/heartbeat
==================================

undefined method `name' for nil:NilClass

> To access an interactive console with this error, point your browser to: /__better_errors


app/views/api/v2/_standard_response.json.jbuilder, line 18
----------------------------------------------------------

``` ruby
   13   json.ignore_nil!
   14   
   15   json.server @server
   16   json.source "#{request.method} #{request.path}"
   17   json.time Time.now.to_formatted_s(:iso8601)
>  18   json.client @client.name
   19   json.code response.status
   20   json.message Rack::Utils::HTTP_STATUS_CODES[response.status]
   21   
   22   if response.status == 200
   23   

App backtrace

* app/views/api/v2/_standard_response.json.jbuilder:18:in `_app_views_api_v___standard_response_json_jbuilder__1455696549825964062_81440'

* app/views/api/v2/error.json.jbuilder:3:in `_app_views_api_v__error_json_jbuilder__3591388735008983784_81420'

* app/controllers/api/v2/base_api_controller.rb:69:in `handle_internal_server_error'

* app/controllers/api/v2/base_api_controller.rb:58:in `handle_exception'

Full backtrace ...

@aaronskiba this is fixed and should be working now

[aaronskiba]

Full backtrace ...

@aaronskiba this is fixed and should be working now

Cool. I can get the 200 via the heartbeat whether I'm signed in or signed in now.

Unfortunately, I think the fix may have introduced other bugs. I am testing the following curl request:

$ curl -H "Authorization: Bearer ${BEARER_TOKEN}"      -H "Accept: application/json"      "http://127.0.0.1:3000/api/v2/plans"

This works and returns the expected JSON response on #3589, which is not up-to-date with these lastest heartbeat changes. However, when I test on this PR, I get the following response:

{
  "source": "GET /api/v2/plans",
  "time": "2025-12-22T12:03:00-07:00",
  "code": 500,
  "message": [
    "There was a problem in the server."
  ]

I also get the following in the rails console:

Processing by Api::V2::PlansController#index as JSON
  Doorkeeper::AccessToken Load (1.3ms)  SELECT "oauth_access_tokens".* FROM "oauth_access_tokens" WHERE "oauth_access_tokens"."token" = $1 LIMIT $2  [["token", "lp5znDvJAOZndbJRWAZ1eeZ_5NR8ZZs_2raswR-YsgQ"], ["LIMIT", 1]]
Exception message: undefined method `include?' for nil:NilClass

        raise Pundit::NotAuthorizedError unless @scopes.include?('read')
                                                       ^^^^^^^^^
  Rendering api/v2/error.json.jbuilder
  Rendered api/v2/_standard_response.json.jbuilder (Duration: 0.7ms | Allocations: 400)
  Rendered api/v2/error.json.jbuilder (Duration: 1.8ms | Allocations: 728)
Completed 500 Internal Server Error in 10ms (Views: 3.8ms | ActiveRecord: 0.9ms | Allocations: 7697)

This must be related to @scopes = doorkeeper_token.scopes.to_a if doorkeeper_token being removed in commit 3645cae

I am also looking at the @application = ApplicationService.application_name change in app/controllers/api/v2/base_api_controller.rb. I like this renaming because it is consistent with app/controllers/api/v1/base_api_controller.rb. However, json.server @server still exists in app/views/api/v2/_standard_response.json.jbuilder. Changing it to json.application @application should fix things and again align with the api/v1 approach.

The assignments to @client and @resource_owner were removed in commit 3645cae. This will need to be addressed as well.

[gjacob24]

Full backtrace ...

@aaronskiba this is fixed and should be working now

Cool. I can get the 200 via the heartbeat whether I'm signed in or signed in now.

Unfortunately, I think the fix may have introduced other bugs. I am testing the following curl request:

$ curl -H "Authorization: Bearer ${BEARER_TOKEN}"      -H "Accept: application/json"      "http://127.0.0.1:3000/api/v2/plans"

This works and returns the expected JSON response on #3589, which is not up-to-date with these lastest heartbeat changes. However, when I test on this PR, I get the following response:

{
  "source": "GET /api/v2/plans",
  "time": "2025-12-22T12:03:00-07:00",
  "code": 500,
  "message": [
    "There was a problem in the server."
  ]

I also get the following in the rails console:

Processing by Api::V2::PlansController#index as JSON
  Doorkeeper::AccessToken Load (1.3ms)  SELECT "oauth_access_tokens".* FROM "oauth_access_tokens" WHERE "oauth_access_tokens"."token" = $1 LIMIT $2  [["token", "lp5znDvJAOZndbJRWAZ1eeZ_5NR8ZZs_2raswR-YsgQ"], ["LIMIT", 1]]
Exception message: undefined method `include?' for nil:NilClass

        raise Pundit::NotAuthorizedError unless @scopes.include?('read')
                                                       ^^^^^^^^^
  Rendering api/v2/error.json.jbuilder
  Rendered api/v2/_standard_response.json.jbuilder (Duration: 0.7ms | Allocations: 400)
  Rendered api/v2/error.json.jbuilder (Duration: 1.8ms | Allocations: 728)
Completed 500 Internal Server Error in 10ms (Views: 3.8ms | ActiveRecord: 0.9ms | Allocations: 7697)

This must be related to @scopes = doorkeeper_token.scopes.to_a if doorkeeper_token being removed in commit 3645cae

I am also looking at the @application = ApplicationService.application_name change in app/controllers/api/v2/base_api_controller.rb. I like this renaming because it is consistent with app/controllers/api/v1/base_api_controller.rb. However, json.server @server still exists in app/views/api/v2/_standard_response.json.jbuilder. Changing it to json.application @application should fix things and again align with the api/v1 approach.

The assignments to @client and @resource_owner were removed in commit 3645cae. This will need to be addressed as well.

Hey @aaronskiba, so sorry about this. I have changed @server to @application, to keep it consistent with V1. I've also put back @scopes and @resource_owner. That was removed unintentionally, so thanks for spotting that. You might still get other errors because we haven't fixed and tested them all, but I think this might be the last change I push before the long break to avoid causing more errors :D

Also, please feel free to commit any changes you feel are necessary (some of which you have already added as comments in this PR).