- Don't throw exception here, rather let the `isSameType(Type, Type)` check handle that
- We still keep the warning we print about missing cases implementation-wise
- Made `BinOpInstr`'s left-hand side instruction of type `Value` and right-hand side instruction of type `Value` too
- Doesn't make sense to use anything but `Value`-based instructions for it
- Previous compilation may have succeeded, meaning ./tlang.out never gets updated and exits fine with 0, but we only use the last commands exit status to check for a pass for a test.
- By setting this if COMPILATION fails then we exit with its code and the test status is set via that
- Cannot use cast as that can return false positives for an all pointer case as all `Pointer`s are `Integer`s
- Added `isPointerType(Type)` to check the above
- Added then also `isIntegralTypeButNotPointer(Type)` which checks for an `Integer` type but excluding if it is a `Pointer`
- Updated the checks in the `BinaryOperator` branch of `typeCheckThing(DNode)` to do this
- Added a new branch which currently throws an exception as it is unimplememted
- This branch (above) is in `attemptCoercion()` and is to handle the coercion of `Integer` to `Pointer` for pointer arithmetic
- When doing the typechecking/codegen for `BinaryOp`, disable the pointer coercion call to `attemptPointerAriehmeticCoercion()`, instead now make calls in those cases they apply, to `typeEnforce()`
- The above stuff is still broken, not yet implemented.
* ⚡️ Feature: Lexer Interface cleanup (#14)
* LexerInterface
Defined the lexer interface
* Parser
- Fixed import for `Token` class
- Removed the token management fields such as `tokens`, `currentToken` and `tokenPtr` as these are now replaced by our `LexerInterface`, `lexer` field which manages this all for us
- Removed constructor which accepts a `Token[]`, now onyl accept a `LexerInterface`
- Removed `nextToken()`, `hasTokens()`, `getCurrentToken()`, `previousToken()`, `setCursor(ulong)` and `getCursor()`.
- The above now are called via the `lexer` instance
Parser (unit tests)
- Migrated to new `LexerInterface`+`BasicLexer` system
- Hoisted out common imports for unit tests into a `version(unittest)`
TypeChecker (unittests)
- Hoisted out common imports for unit tests into a `version(unittest)`
- Migrated to new `LexerInterface`+`BasicLexer` system
LexerInterface
- Moved to new `lexer.core` package
- Documented module and class
Commands
- Fixed imports for the (now) `BasicLexer`
- Fixed imports for the (now) `lexer.core` package
Compiler
- Fixed imports for the (now) `BasicLexer`
- Use `LexerInterface` instead of `Lexer`
- The `doLex()` method now uses an instance of `BasicLexer` and then downcasts to quickly call `performLex()` in order to tokenize and make them available
- The `doParse()` method now takes in an instance of `LexerInterface` rather than `Token[]`
BasicLexer (previously Lexer)
- Moved to the `lexer.kinds` package
- Now implements `LexerInterface`
- Documented module and class
- Documented the `LexerInterface` methods
Exceptions
- Moved to the `lexer.core` package
- Fixed import of `Token` class
- Now uses `LexerInterface`
Core.Lexer.Package
- Documented package module
Tokens
- Moved to the `lexer.core` package
- Documented module and class
Check
- Fixed import for `Token`
- Fixed import for `BasicLexer`
* `core.lexer` (package)
- Documented all public imports
* Exceptions
- Documented the module
- Documented `LexerError` and its members
- Documented `LexerException`, its members too
* Tokens
- Documented the fields (using proper syntax)
- Documented constructor and methods
* BasicLexer
- Removed now-completed TODO
- Added (for clarity) `override` keywords to the `getLine()` and `getColumn()` methods
- Moved `getLine()`, `getColumn()` and `getTokens()` altoghether
- Made `getTokens()` override-marked
- Documented `getTokens()`
* Check
- Removed weird TODO that makes no sense
- Documented some of the members of `SymbolType`
* Check
- Documented a few more enum members of `SymbolType`
- Fixed documentation (and added a TODO) for the `SymbolType.LE_SYMBOL`
* Check
- Documented a few more enum members of `SymbolType`
* Check
- Documented `isType(string)`
- Added a TODO for `isTYpe(string)` to "Check if below is even used
- Documented `isPathIdentifier(string)`
* Check
- Updated description of `isPathIdentifier(string)` to note it can contain underscores
- Documented isIdentifier(string)`
- Updated `SymbolType.IDENT_TYPE` to acknowledge underscores
- Documented `isAccessor(Token token)` and `isModifier(Token)`
* Check
- Documented `isIdentifier_NoDot(Token tokenIn)`, `isIdentifier_Dot(Token tokenIn)`, `isNumericLiteral(string token)`
- Removed uneeded import of `BasicLexer`
- Moved import to the top of file
* Check
- Documented `getSymbolType(Token tokenIn)`, `isMathOp(Token token)`, `isBinaryOp(Token token)`
* Check
- Documented the `symbols.check` module
* Builtins
- Properly documented `getBuiltInType(TypeChecker, string)`
* Builtins
- Documented module
* Typing (core)
- Documented module
- Documented all members
* Exceptions (lexer)
- Fixed documentation missing parameters
* Check
- Make comments docs/ddox compatible
* BasicLexer
- Fixed parameter name in documentation
* BasixLexer
- Fixed formatting in documentation for class
* Typing (core)
- Documented all remaining class members and fields
* TypeChecker
- `typeEnforce()` now will not change the type of `Value`-based instruction `v2` but rather return, on successful coercion set a `ref`-based argument to a new instance of a `CastedValueInstruction`, if coercion fails or was disabled and types mismatched then an exeption is thrown as normal.
- If the types are an exact same match, a-la `isSameType(Type, Type)`, then this `ref` value is set to `v2` (makes programming easy) else we would have no way to know
- `attemptCoerce()` now, to go with the above changes to `typeEnforce()`, returns a `CatsedValueInstruction` to the to-type on successful coercion, else an exception is thrown as usual
- Updated two cases of `typeEnforce()` usage to the new method signature, also now add a sanity check assertion that the types now DO match as they should
* TypeChecker
- We need not set it again, look the value we use when we CALL `typeEnforce()` is that of the `fromInstruction` and if no changes occur we still have it, it is fine - if it changes via the call to `typeEnforce()` via the `ref` based argument thne same old
- No need for us to set it here in the event of no changes, we are writing back the exact same Instruction/object-reference
* TypeChecker (unit tests)
- Upgraded to the new `typeEnforcer()` method signature
- Disabled invalid unit test (marked for re-writing)
- I should re-write the below. It is now incorrect as I DO ALLOW coercion of non literal-based instructions now - so it fails because it is using an older specification of TLang
- If the to-type and provided-type are both numerical then use a size-based test
Test cases
- Added two test cases which test `typeEnforce()` on incoming `Value`-based instructions as part of variable declarations
- Variable declarations (with assignments) now use the `typeEnforce()` method with coercion allowed in order to do the type checking and coercion changes
- Added a comment explaining a certain branch of `attemptCoercion(Type, Value)`
* Parser
- Added new interface `Cloneable`
* Symbols
- Added new `Statement`-based type: `Macro` to support `Macro`(s)
* MetaProcessor
- Added the `MetaProcessor`
TypeChecker
- Added the `MetaProcessor` instance to the `TypeChecker`, it will be instantiated upon the `TypeChecker`'s construction and later have its `.process()` method called as the first call in `beginCheck()`
* TypedEntity
- Added a `setType(string)` method to update the internal `type` field
* MetaProcessor
- Added a type-re-writing facility via `typeRewrite(TypedEntity)` which will re-write the types such as `size_t`, `ssize_t` and so forth
Test cases
- Added a test case `meta/types.t` which tests this
* MetaProcessor
- Updated the constructor to only take in an instance of the `TypeChecker`
- Updated the `process()` method to take in a `Container` such that it can be used recursively
- Commented code
- Added a recursive call to `process(Container)` when `curStmt` is a kind-of `Container` (this ensures that we reach the `VariableParameter`s of `Function` (die to them making up the `Statement[]` of `Function` type)
- Removed unused `cmp` import `std.string`
- Added type-rewrite for `ssize_t` -> `long`
TypeChecker
- Updated the constructor call to `MetaProcessor` to use its new API
- Updated call to `process()` to now be `process(modulle)`
Test cases
- Updated `types.t` to test re-writing of the `Function`'s parameters
* MetaProcessor
- Added another FIXME
* Mcro
- Added interface `MTypeRewritable` to represent any AST node which has a `setType(string)` and `string getType()` method for rewriting- Added `Sizeof` which is a kind-of `IntegerLiteral`
Data
- Made `TypedEntity` implement the `MTypeRewritable` interface
Expressions
- Made `IntegerLiteral` non-final such that we can inherit from it
- Added a final `setNumber(string)` method to `NumberLiteral` to update the literal
MetaProcessor
- The type rewriting mechanism now operates on `MTypeRewritable` AST nodes
- Work has begun on `sizeOf_Literalize(Sizeof)` which is to determine the `Type` of the `Sizeof` statement and then calculate the memory width and update its literal (as it is a kind-of `NunberLiteral`) to said size
Test cases
- Added `meta/sizeof.t` to test `sizeof` functionality
* Mcro
- Added interface type `MStatementSearchable`
* Mcro
- Redefined `MStatementSearchable`
- Added `MStatementReplaceable`
- Added `Repr` (a kind-of `Expression`) which will be used as an example to test out the aforementioned two interfaces
* MStatementSearchable
- Added method `search(TypeInfo_Class clazzType)` which uses a `TypeInfo_Class` to search for all types matching that (and which are sub-types of `Statement` and then adds these to a list and returns it in the form of `Statement[]`
* MStatementReplaceable
- Implemented `replace(Statement thiz, Statement that)` which replaces the `Statement` in the first argument with that of the `Statement` in the second argument
* MStatementSearchable
- Added documentation for `search(TypeInfo_Class)` method
* Mcro
- Made `Repr` implement `MStatementSearchable`
- Added new interface `MCloneable` to represent PNode's which are deeply-cloneable
* Data
- Made `DiscardStatement` searchabe via implementing `MStatementSearchable`
- Added a stub override for implementing `MStatementReplaceable` in `DiscardStatement`
* Mcro
- Updated `MStatementReplaceable` to have its `replace(Statement, Statement)` method return a boolean `true` if the replacement was successful, else `false`
* Parsing
- Added the ability to parse a `Repr` statement
Check
- Added `GENERIC_TYPE_DECLARE` and `REPR` as new `SymbolType`(s)
Data
- `DiscardStatement` now implements the `bool replace(Statement, Statement)` method
MetaProcessor
- Added the ability to replace statements that occur within any _other_ statements which implement `MStatementSearchable` and `MStatementReplaceable`
Test cases
- Added `meta/simple_meta_replace.t` to test all of this
Diagrams
- Added diagram on how the meta system works
* MetaProcessor
- Removed unused `replace` method
* MetaProcessor
- Accept a new argument to the `MetaProcessor(TypeChecker)` constructor indicating whether or not the `MetaProcessor` is enabled or not
TypeChecker
- Enable the `MetaProcessor`
* Mcro
- Removed `Sizeof`, we will let it be parsed as a normal `FunctionCall` and then inspect in `MetaProcessor`
* MetaProcessor
- Disabled `sizeOf_Literalize` for now
- Search for all `FunctionCall` statements in `process(Container)`
* MetaProcessor
- Removed old code for testing `MStatementSearchable` and `MStatementReplaceable`
- Added note that we will have to investigate a recursive `MTypeRewritable` to be able to support things like `sizeof(size_t)`
- Implemented module-level `sizeof(<ident_type>)` support
- Added `Number`-kind of types support to `sizeOf_Literalize`(string)`
Containers (`Module`)
- Added `MStatementSearchable` and `MStatementReplaceable` support to `Module` container type
Data
- Added `MStatementSearchable` and `MStatementReplaceable` support to `Variable`
- Added `MStatementSearchable` and `MStatementReplaceable` support to `VariableAssignment`
- Work-in-progress for adding `MStatementSearchable` and `MStatementReplaceable` support to `FunctionCall`
- Added a note to return `false` in `DiscardStatement` if the statement to be replaced is us (the `DiscardSTatement`) ourselves
Test cases
- Updated the `meta/sizeof.t` test case
* Containers
- Inherit from `MStatementSearchable` and `MStatementReplaceable`
- Removed direct interface inheritance of `MStatementSearchable, MStatementReplaceable` and `MStatementReplaceable`, rely on `Container` now
* Struct
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
Clazz
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
BinaryOperatorExpression
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
Function
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
Variable
- Fixed the `replace` method implementation which had a bug that would never replace the node `VariableAssignment` inside of it
VariableAssignmentStdAlone
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
IfStatement
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
WhileLoop
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
ForLoop
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
Branch
- Implemented the `search` method for `MStatementSearchable`
- Implemented the `replace` method for `MStatementReplaceable`
* MetaProcessor
- Added a future TODO for occurence of certain types that are alises (i.e. `size_t` appearing in a expression context like `sizeof(size_t)`
- Now that all `Container`-based types are `MStatementReplaceable` we no longer need this check
* MetaProcessor
- Removed `break` which caused only one `sizeof()` function call to be replaced, we should have been looping over each `FunctionCall` found with `search` and then replacing ech that had a name of `sizeof` with a `NumberLiteral` expression
- Moved all type alias replacement code into a new method; `doTypeAlias(Container, Statement)`
- Added some rudiementary support for replacing any `IdentExpression` containing `size_t` with `uint`
IdentExpression
- Made it `MStatementSearchable` and `MStatementReplaceable`
Test cases
- Updated test case `meta/sizeof.t` to test the `sizeof(<expr>)` AST node in the `MetaProcessor`
Documentation
- Added diagram showing the `MStatementSearchable` and `MStatementReplaceable` in action
* Compiler
- If the T compiler was built on `X86` then set the maximum width to 4 bytes
- If the T compiler was built on `X86_64` then set the maximum width to 8 bytes
- Pass in the `Compiler` instance to the `TypeChecker` in `doTypeCheck()`
TypeChecker
- Added `Compiler` field and now accept it in constructor
- If just single parameter constructor then pass in `null` as the `Compiler` instance
- Added `getCompiler()` to get the instance
MetaProcessor
- Extract the `CompilerConfiguration` via the `TypeChecker`'s `getCompiler()`
- Implemented `getSystemType(string)` which will map `size_t`/`ssize_t` to the correct maximum width'd type your system supports via the `types:max_width` config entry
* CompilerConfiguration
- Added `defaultConfig()`
* Compiler
- Removed `defaultConfig()`
- During construction call `CompilerConfiguration.defaultConfig()` in order to generate the default config
- Pass the config into `TypeChecker` in `doTypeCheck()`
* TypeChecker
- No longer store a field for the `Compiler` but rather store a field for the `CompilerConfiguration`
- Removed single parameter constructor for `TypeChecker`
- Constructor now uses the default `CompilerConfiguration` if not specified; therefore fixing the issue with unit tests failing
MetaProcessor
- Extract the compiler configuration via `tc.getConfig()`
- Updated `typeRewrite(MTypeRewritable)` to use `getSystemType(string)` to lookup concrete types for `size_t`/`ssize_t`
- `doTypeAlias(Container, Statement)` now uses `getSsstemType(string)` to lookup the concrete types for alises such as `size_t`/`ssize_t`
* MetaProcessor
- Implemented `isSystemType(string)` which returns `true` if the provided type is a system type alias (such as `size_t` or `ssize_t`), `false` otherwise
* MetaProcessor
- Implemented `isTypeAlias(string)` which determines if the given type is a type alias.
- Implemented `getConcreteType(string typeAlias)` which transforms the type alias into its
concrete type; this method incorporates defensive programming in that it will only apply the transformation IF
the provided type alias is infact a type alias, otherwise it performs an identity transformation
and returns the "alias" untouched.
* TypeChecker
- Clean up
* MetaProcessor
- `doTypeAlias(Container, Statement)` now makes use of `isTypeAlias(string)` and `getConcreteType(string)`
* - `typeRewrite(MTypeRewritable)` now makes use of `isTypeAlias(string)` and `getConcreteType(string)`
* MetaProcessor
- Cleaned up
* MetaProcessor
- Removed unused import
* MetaProcessor
- Removed now-completed TODO
* MetaProcessor
- Updated comment before call to `doTypeAlias(Container, Statement)`
* Containers
- `Module` now applies re-parenting in its `replace()`
- `Struct` now applies re-parenting in its `replace()`
- `Clazz` now applies re-parenting in its `replace()`
Data
- `Function` now applies re-parenting in its `replace()`
* Test cases
- Added `simple_template_type_def.t` for future test cases
* - Updated `.gitignore`
* Check
- Removed `SymbolType.REPR` which was used for testing early versions of the `MetaProcessor`
* Mcro
- Removed `Repr` which was used for testing in the early stages of `MetaProcessor`
* Check
- Removed reference to `SymbolType.REPR` in checker
* Parser
- Removed reference to `SymbolType.REPR` and `Repr` which was used for testing the `MetaProcessor` in early stages
* Test cases
- Added `simple_direct_func_call.t` to test direct function calls
* Test cases
- Removed tabs which broke lexing
* AST nodes
- `FunctionCall` now has the ability to be marked as statement-level by calling `makeStatementLevel()`, this can then be queried later via `isStatementLevelFuncCall()`
* Parser
- Statement-level function calls were never actually returned, resulting in `null` being returned by `parseName()` - this has now been fixed.
- Along with this we now "mark" this `FunctionCall` AST node as statement-level when it occurs in `parseName()`
* Instruction
- Allow `FuncCallInstr` to be makred as statement-level and queired much in the same manner as its corresponding AST-node/parser-node `FunctionCall`
* Dependency
- Added support for `DNode` generation in `generalPass()` for `FunctionCall` AST nodes
* TypeChecker
- Handle `FunctionCall`s differently in terms of code generation dependent on whether or not rhe call is within an expression of statement-level
* DGen
- Handle statement-level function calls (`FuncCallInstr`s) differently by tacking on an additional `";"` to the emit string
* - Added `simple_direct_func_call.t` to GitHub pipeline
* DGen
- Added instrumentation for semantic code generation for `simple_function_recursion_factorial.t`
- Added TODO for future `simple_func_call_direct.t`
Test cases
- Added `simple_function_recursion_factorial.t` to test recursion
Pipelines
- Added `simple_function_recursion_factorial.t` to `emit` stage
* DGen
- Made `if` an `else if` - this wouldn't of affected anything but just to be correct
* DGen
- Added semantic code generation instrumentation for test case `simple_direct_func_call.t`
Test cases
- Updated test case `simple_direct_func_call.t`
