Skip to content
Snippets Groups Projects
Commit 703cb8eb authored by michael.minelli's avatar michael.minelli
Browse files

Wiki => Move wiki on the dojo documentation website

parent d1bdb1a0
Branches
Tags
1 merge request!3Return error when client headers are missing (issue #19)
Pipeline #29873 passed
Stage: code_quality
Stage: test
Hide whitespace changes
Inline Side-by-side
README.md
+ 2
− 38
View file @ 703cb8eb
# DojoBackendAPI # Documentation of `The Dojo Backend API`
## Development environment All documentations are available on the [Dojo documentation website](https://www.hepiapp.ch/) : https://www.hepiapp.ch/.
\ No newline at end of file
### Env vars
You can decrypt env var stored in the `.env.vault` file with the following commands in the `ExpressAPI` folder:
```bash
> npx dotenv-vault local keys
environment DOTENV_KEY
─────────── ─────────────────────────────────────────────────────────────────────────
development dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development
Set DOTENV_KEY on your server
> npx dotenv-vault local decrypt dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development > .env.development
```
**The `.env.keys` file have to be requested to the project maintainers.**
### Database
For development, you can use the docker-compose file in the `Resources/DevInfra/` folder.
```bash
docker-compose -f Resources/DevInfra/docker-compose.yml up -d
```
This will run a MariaDB database on port `59231` with the following credentials: `root:9buz7f312479g6234f1gnioubervw79b8z`
A second container is created with the Adminer tool on port `62394`.
#### Structure creation and seeding
The following command will create the database structure and seed it with some exemple data.
```bash
npm run database:deploy:dev
```
\ No newline at end of file
Wiki/Development/1-How-to-setup-your-development-environment.md deleted 100644 → 0
+ 0
− 131
View file @ d1bdb1a0
# How to setup your development environment
## Introduction
This tutorial describes how to setup your development environment for building the Dojo API by detailing the
prerequisites and dependencies needed.
## Technologies
The API is built using [NodeJS](https://nodejs.org/en/) and [NPM](https://www.npmjs.com/).
The programming language used is [Typescript](https://www.typescriptlang.org/) v5.
## Prerequisites
In order to build the API you will need the following tools:
- [NodeJS](https://nodejs.org/en/) (version 20 or higher)
- [NPM](https://www.npmjs.com/) (version 10 or higher)
Install NodeJS and NPM by following the instructions on the [official website](https://nodejs.org/en/download/package-manager).
Or via Node Version Manager (NVM) by following the instructions on the [official website](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).
## Dependencies
Here are the main dependencies used by the API (you do not need to install them manually or globally on your system):
- [Axios](https://www.npmjs.com/package/axios): a promise-based HTTP client for the browser and Node.js. It is
used to make HTTP(S) requests to the Dojo backend and Gitlab.
- [Dotenv](https://www.npmjs.com/package/dotenv): used to load environment variables from a .env file.
- [Dotenv-vault](https://www.npmjs.com/package/dotenv-vault): a CLI to sync .env files across machines,
environments, and team members.
- [Express](https://www.npmjs.com/package/express): a minimal and flexible Node.js web application framework that
provides a robust set of features for web and mobile applications. Used to create the API server.
- [express-validator](https://www.npmjs.com/package/express-validator): used to validate and sanitize express requests.
- [Morgan](https://www.npmjs.com/package/morgan): used to log HTTP requests.
- [JsonWebToken](https://www.npmjs.com/package/jsonwebtoken): used to generate and validate [JSON Web Tokens](https://jwt.io/).
- [Prisma](https://www.npmjs.com/package/prisma): a modern database access & ORM for Node.js. Used to construct (from
scratch and migrations) and interact with the database.
- [Winston](https://www.npmjs.com/package/winston): used for logging purposes. Used in combination with Morgan to
log HTTP requests in the format defined with Winston.
- [zod](https://www.npmjs.com/package/zod): a TypeScript-first schema validation with static type inference. Used
in the projet to validate json files created by the user.
## Installation
First of all, you need to clone the repository:
```bash
$ git clone --recurse-submodule https://gitedu.hesge.ch/dojo_project/projects/backend/dojobackendapi.git
```
Then, you need to move to the project's directory:
```bash
$ cd dojobackendapi/ExpressAPI
```
To install the dependencies listed above you can use the following command in the base directory of the project:
```bash
$ npm install
```
## Environment variables
Environment variables are used to store sensitive information such as API keys, passwords, etc.
They are also used to store configuration information that can be changed without modifying the code.
### Using on your development machine
To use environment variables on your development machine, you need the `.env.keys` file in addition of the `.env.vault`
file present in the repository.
**The `.env.keys` file have to be requested to the project maintainer: [Michaël Minelli](mailto:dojo@minelli.me).**
### Decrypting the environment variables
You can decrypt env var stored in the `.env.vault` file with the following commands in the project's main folder.
Here is an example of how to decrypt the environment variables for the development environment:
```bash
$ npx dotenv-vault local keys
environment DOTENV_KEY
─────────── ─────────────────────────────────────────────────────────────────────────
development dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development
Set DOTENV_KEY on your server
$ npx dotenv-vault local decrypt 'dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development' > .env.development
```
### How to modify the environment variables
You can modify environment variables in `.env` files that you have previously decrypted.
You also need to re-encrypt the modified `.env` files to store them in the `.env.vault` with the following command:
```bash
$ npm run dotenv:build
```
## Database
For the development, you can use the docker compose file in the `Resources/DevInfra/` folder.
The following command have to be executed in the base directory of the repository:
```bash
$ docker compose -f Resources/DevInfra/docker-compose.yml up -d
```
This will run a MariaDB database on port `59231` with the following credentials: `root:9buz7f312479g6234f1gnioubervw79b8z`
A second container is created with the Adminer tool on port `62394`.
#### Structure creation and seeding
The following command (to be executed from the `ExpressAPI` folder) will create the database structure and seed it with
some example data.
```bash
$ npm run database:deploy:dev
```
## Run the API
To run the API (in dev mode) you can use the following command in the base directory of the project:
```bash
$ npm run start:dev
```
Wiki/Development/2-How-to-add-a-new-route.md deleted 100644 → 0
+ 0
− 410
View file @ d1bdb1a0
# How to add a new route to the Dojo API
## Introduction
This tutorial describes how to add a new route to the Dojo API. For that we take two existing routes and describe
his implementation. This route allow a member of the teaching staff of an assignment to publish / unpublish it.
## Prerequisites
All the prerequisites are described in
[How to setup your development environment](1-How-to-setup-your-development-environment) tutorial.
## Properties of the new route
### Description of the route
- `Route :` /assignments/:assignmentNameOrUrl/publish
- `Verb :` PATCH
- `Resume :` Publish an assignment
- `Protection type :` Clients_Token
- `Protection :` TeachingStaff of the assignment or Admin role
### Params of the request (url params)
- `Name :` assignmentNameOrUrl
- `Description :` The name or the url of an assignment.
- `Location :` Query
- `Required :` Yes
- `Data type :` string (path)
### Possible Response(s)
- `Code :` 200
- `Description :` OK
- `Content of the response :`
```json
{
"timestamp": "1992-09-30T19:00:00.000Z",
"code": 200,
"description": "OK",
"sessionToken": "JWT token (for content, see schema named 'SessionTokenJWT')",
"data": {}
}
```
- `Code :` 401 - Standard
- `Code :` 404 - Standard
## Routes files structure
The routes files are located in the `src/routes` folder. All routes files are named with the following pattern:
`SubjectRoutes.ts` where `Subject` has to be replaced by the general subject of the routes implemented in it (f.e.
Exercise, Assignment, Session, etc.).
### Application to our use case
In our case we will add our route to the file with the following path :
`src/routes/AssignmentRoutes.ts`.
## Routes class inheritance
All routes files must inherit from the `RoutesManager` interface. This interface is located in the
`src/express/RoutesManager.ts` file.
When you inherit from this interface you will need to implement the following method :
- `registerOnBackend(backend: Express): void;`: This method get the express backend object for register new routes.
### Apply to our use case
Now, the `src/routes/AssignmentRoutes.ts` file will look like this:
```typescript
import RoutesManager from '../express/RoutesManager';
class AssignmentRoutes implements RoutesManager {
protected commandName: string = 'publish';
registerOnBackend(backend: Express) {
...
backend.patch('/assignments/:assignmentNameOrUrl/publish', ..., this.publishAssignment.bind(this));
backend.patch('/assignments/:assignmentNameOrUrl/unpublish', ..., this.unpublishAssignment.bind(this));
}
protected async commandAction(assignmentNameOrUrl: string, options: { force: boolean }): Promise<void> {
private async publishAssignment(req: express.Request, res: express.Response) {
return this.changeAssignmentPublishedStatus(true)(req, res);
}
private async unpublishAssignment(req: express.Request, res: express.Response) {
return this.changeAssignmentPublishedStatus(false)(req, res);
}
private changeAssignmentPublishedStatus(publish: boolean): (req: express.Request, res: express.Response) => Promise<void> {
return async (req: express.Request, res: express.Response): Promise<void> => {
...
};
}
}
export default new AssignmentRoutes();
```
## Define request param binding
The goal of the step is to define a generic request parameter and bind functions that will complete a property in the
`request.boundParams` object.
For exemple, we can define a function that will take the `assignmentNameOrUrl` parameter and find the assignment
corresponding to it, add it to the request object and call the next function.
Steps :
1. First of all, we need to add the new property to the type located in the `src/types/express/index.d.ts` file in the
boundParams object with type: `Type | undefined` with `Type` is the type of the param.
2. In the `src/middlewares/ParamsClallbackManager.ts` file we need to:
1. Add the new property to the `boundParams` object in the `initBoundParams` method with the `undefined` value.
2. Call the the `listenParam` method in the `registerOnBackend` method for each paramer as follows:
```typescript
this.listenParam(paramName: string, backend: Express, getFunction: GetFunction, arrayOfArgsForGetFunction: Array<unknown>, boundParamsIndexName: string
### Application to our use case
1. In the `src/types/express/index.d.ts` file:
```typescript
declare global {
namespace Express {
interface Request {
...
boundParams: {
...
assignment: Assignment | undefined
};
}
}
}
```
2. In the `src/middlewares/ParamsClallbackManager.ts` file:
```typescript
...
initBoundParams(req: express.Request) {
if ( !req.boundParams ) {
req.boundParams = {
...,
assignment: undefined
};
}
}
registerOnBackend(backend: Express) {
...
this.listenParam('assignmentNameOrUrl', backend, (AssignmentManager.get as GetFunction).bind(AssignmentManager), [ {
exercises: true,
staff : true
} ], 'assignment');
}
```
## Add security to the route
A middleware is available to check permissions. It is located in the `src/middlewares/SecurityMiddleware.ts` file.
This file does not need to be edited unless you want to add some new security tests.
You can use the function `SecurityMiddleware.check(checkIfConnected: boolean, ...checkTypes: Array<SecurityCheckType>)`
function as a middleware in the route definition.
You have the possibility to just check if the user is connected with the first parameter. If you want to add a more
specific check you can add some parameters with the `SecurityCheckType` enum value (f.e. teaching staff, assignment
staff, etc.).
**WARNING:** The `SecurityCheckType` args array is interpreted as an `OR` condition. So if you call :
`SecurityMiddleware.check(true, SecurityCheckType.TEACHING_STAFF, SecurityCheckType.ASSIGNMENT_STAFF)`, the middleware
will check if the user is connected and if he have the teaching staff role or in the assignment staff.
### Application to our use case
For our routes we want to test if the user is connected and if he is in the staff of the assignment. So we will
complete the routes definitions like this:
```typescript
registerOnBackend(backend: Express) {
...
backend.patch('/assignments/:assignmentNameOrUrl/publish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.publishAssignment.bind(this));
backend.patch('/assignments/:assignmentNameOrUrl/unpublish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.unpublishAssignment.bind(this));
}
```
## Control the request body
The Dojo API use the `express-validator` (based on `validator.js`) library to validate and sanitize the request body of
requests. We use the last version of the library that is `7.0.1`. The documentation is available at the following link:
[express-validator](https://express-validator.github.io/docs/).
### Application to our use case
This tutorial will not go deeper in the `express-validator` library because we do not need have request body in our
routes. But if you need to use it, you can find examples in `src/routes/ExerciseRoutes.ts` or
`src/routes/AssignmentRoutes.ts` file (look at the `ExpressValidator.Schema` objects and their usage).
## Define the action
When you define an action you define a function with the following signature:
```typescript
(req: express.Request, res: express.Response): Promise<void>
```
To respond to the request you need to use the following method (works even if the user is not connected):
```typescript
return req.session.sendResponse(res: express.Response, code: number, data?: unknown, descriptionOverride?: string, internalCode?: number)
```
Where :
- `res` is the express response object
- `code` is the HTTP status code of the response
- `data` is the data to send in the response (it must be a JSON serializable value)
- `descriptionOverride` is a field that you can use to give a custom string description of the response (by default it
will be the description of the HTTP status code)
- `internalCode` is the internal code of the response (if you want to add a custom internal code to be more specific
than the HTTP status code)
### Application to our use case
For the implementation we will split our code into four parts:
1. If we want to publish, we check first if it is possible (if the last pipeline status is `success`).
2. Change the project visibility on Gitlab.
3. Change the assignment published status on the Dojo database.
4. Send the response to the client.
```typescript
// Part 1
if ( publish ) {
const isPublishable = await SharedAssignmentHelper.isPublishable(req.boundParams.assignment!.gitlabId);
if ( !isPublishable.isPublishable ) {
return req.session.sendResponse(res, StatusCodes.BAD_REQUEST, { lastPipeline: isPublishable.lastPipeline }, isPublishable.status?.message, isPublishable.status?.code);
}
}
try {
// Part 2
await GitlabManager.changeRepositoryVisibility(req.boundParams.assignment!.gitlabId, publish ? GitlabVisibility.INTERNAL : GitlabVisibility.PRIVATE);
// Part 3
await db.assignment.update({
where: {
name: req.boundParams.assignment!.name
},
data : {
published: publish
}
});
// Part 4
req.session.sendResponse(res, StatusCodes.OK);
} catch ( error ) {
if ( error instanceof AxiosError ) {
res.status(error.response?.status ?? HttpStatusCode.InternalServerError).send();
return;
}
logger.error(error);
res.status(StatusCodes.INTERNAL_SERVER_ERROR).send();
}
```
## Documentation
Finally, the fun part. We need to document our new created routes. For that we use the [`OpenAPI` specification in his
3.1.0 version](https://express-validator.github.io/docs/).
The documentation is yaml formatted and is in the `assets/OpenAPI/OpenAPI.yml` file.
## Use case: final code
### AssignmentRoutes.ts
```typescript
import { Express } from 'express-serve-static-core';
import express from 'express';
import { StatusCodes } from 'http-status-codes';
import RoutesManager from '../express/RoutesManager';
import SecurityMiddleware from '../middlewares/SecurityMiddleware';
import SecurityCheckType from '../types/SecurityCheckType';
import GitlabManager from '../managers/GitlabManager';
import { AxiosError, HttpStatusCode } from 'axios';
import logger from '../shared/logging/WinstonLogger';
import db from '../helpers/DatabaseHelper';
import GitlabVisibility from '../shared/types/Gitlab/GitlabVisibility';
import SharedAssignmentHelper from '../shared/helpers/Dojo/SharedAssignmentHelper';
class AssignmentRoutes implements RoutesManager {
registerOnBackend(backend: Express) {
backend.patch('/assignments/:assignmentNameOrUrl/publish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.publishAssignment.bind(this));
backend.patch('/assignments/:assignmentNameOrUrl/unpublish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.unpublishAssignment.bind(this));
}
private async publishAssignment(req: express.Request, res: express.Response) {
return this.changeAssignmentPublishedStatus(true)(req, res);
}
private async unpublishAssignment(req: express.Request, res: express.Response) {
return this.changeAssignmentPublishedStatus(false)(req, res);
}
private changeAssignmentPublishedStatus(publish: boolean): (req: express.Request, res: express.Response) => Promise<void> {
return async (req: express.Request, res: express.Response): Promise<void> => {
if ( publish ) {
const isPublishable = await SharedAssignmentHelper.isPublishable(req.boundParams.assignment!.gitlabId);
if ( !isPublishable.isPublishable ) {
return req.session.sendResponse(res, StatusCodes.BAD_REQUEST, { lastPipeline: isPublishable.lastPipeline }, isPublishable.status?.message, isPublishable.status?.code);
}
}
try {
await GitlabManager.changeRepositoryVisibility(req.boundParams.assignment!.gitlabId, publish ? GitlabVisibility.INTERNAL : GitlabVisibility.PRIVATE);
await db.assignment.update({
where: {
name: req.boundParams.assignment!.name
},
data : {
published: publish
}
});
req.session.sendResponse(res, StatusCodes.OK);
} catch ( error ) {
if ( error instanceof AxiosError ) {
res.status(error.response?.status ?? HttpStatusCode.InternalServerError).send();
return;
}
logger.error(error);
res.status(StatusCodes.INTERNAL_SERVER_ERROR).send();
}
};
}
}
export default new AssignmentRoutes();
```
### src/types/express/index.d.ts
```typescript
import Session from '../../controllers/Session';
import { Assignment, Exercise } from '../DatabaseTypes';
// to make the file a module and avoid the TypeScript error
export {};
declare global {
namespace Express {
export interface Request {
session: Session,
boundParams: {
assignment: Assignment | undefined, exercise: Exercise | undefined
}
}
}
}
```
### ParamsCallbackManager.ts
```typescript
import { Express } from 'express-serve-static-core';
import express from 'express';
import { StatusCodes } from 'http-status-codes';
import AssignmentManager from '../managers/AssignmentManager';
type GetFunction = (id: string | number, ...args: Array<unknown>) => Promise<unknown>
class ParamsCallbackManager {
protected listenParam(paramName: string, backend: Express, getFunction: GetFunction, args: Array<unknown>, indexName: string) {
backend.param(paramName, (req: express.Request, res: express.Response, next: express.NextFunction, id: string | number) => {
getFunction(id, ...args).then(result => {
if ( result ) {
this.initBoundParams(req);
(req.boundParams as Record<string, unknown>)[indexName] = result;
next();
} else {
req.session.sendResponse(res, StatusCodes.NOT_FOUND, {}, 'Param bounding failed: ' + paramName);
}
});
});
}
initBoundParams(req: express.Request) {
if ( !req.boundParams ) {
req.boundParams = {
assignment: undefined,
exercise : undefined
};
}
}
registerOnBackend(backend: Express) {
this.listenParam('assignmentNameOrUrl', backend, (AssignmentManager.get as GetFunction).bind(AssignmentManager), [ {
exercises: true,
staff : true
} ], 'assignment');
...
}
}
export default new ParamsCallbackManager();
```
\ No newline at end of file
Wiki/Home.md
+ 2
− 16
View file @ 703cb8eb
# Documentation of the `dojo` API # Documentation of `The Dojo Backend API`
In this wiki you will find the documentation related to the `dojo` API. All documentations are available on the [Dojo documentation website](https://www.hepiapp.ch/) : https://www.hepiapp.ch/.
\ No newline at end of file
## Dojo Project
The dojo platform is an online tool built to help practice programming by allowing users to propose assignments and perform them as exercises.
The two major concepts of the platform are the **assignments** (provided by teaching staff) and the **exercises** (performed by students).
More details here : [Dojo detailed presentation](0-Dojo-presentation)
## Development / Contribution
* [How to contribute]() - Available soon
* [How to setup your development environment](Development/1-How-to-setup-your-development-environment)
* [How to add a new route](Development/2-How-to-add-a-new-route)
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment